2015-07-14 16:14:31.515863 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:31.517419 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 2015-07-14 16:14:31.517460 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 2015-07-14 16:14:31.517482 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set 2015-07-14 16:14:31.517497 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 21 0x4a97b00 mdsbeacon(94247/codetest up:reconnect seq 6 v513) v3 2015-07-14 16:14:31.517534 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:31.517549 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 21 2015-07-14 16:14:31.517564 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:31.517582 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 21 ==== mdsbeacon(94247/codetest up:reconnect seq 6 v513) v3 ==== 120+0+0 (401768720 0 0) 0x4a97b00 con 0x4a3e160 2015-07-14 16:14:31.517620 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 2015-07-14 16:14:35.515829 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 7 v513) v3 -- ?+0 0x4a96f00 con 0x4a3e160 2015-07-14 16:14:35.515894 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:35.515997 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:35.517577 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 2015-07-14 16:14:35.517623 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 2015-07-14 16:14:35.517645 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set 2015-07-14 16:14:35.517659 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 22 0x4a97800 mdsbeacon(94247/codetest up:reconnect seq 7 v513) v3 2015-07-14 16:14:35.517698 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:35.517729 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 22 2015-07-14 16:14:35.517750 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:35.517736 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 22 ==== mdsbeacon(94247/codetest up:reconnect seq 7 v513) v3 ==== 120+0+0 (3910525111 0 0) 0x4a97800 con 0x4a3e160 2015-07-14 16:14:35.517776 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 2015-07-14 16:14:38.031639 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:38.031696 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_keepalive2 14 2015-07-14 16:14:38.031695 2015-07-14 16:14:38.031751 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:39.515916 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 8 v513) v3 -- ?+0 0x4a96000 con 0x4a3e160 2015-07-14 16:14:39.515994 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:39.516095 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:39.517857 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 2015-07-14 16:14:39.517904 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 2015-07-14 16:14:39.517952 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set 2015-07-14 16:14:39.517968 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 23 0x4b3c900 mdsbeacon(94247/codetest up:reconnect seq 8 v513) v3 2015-07-14 16:14:39.518009 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:39.518040 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 23 2015-07-14 16:14:39.518061 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:39.518058 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 23 ==== mdsbeacon(94247/codetest up:reconnect seq 8 v513) v3 ==== 120+0+0 (4106708175 0 0) 0x4b3c900 con 0x4a3e160 2015-07-14 16:14:39.518094 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 2015-07-14 16:14:43.516003 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 9 v513) v3 -- ?+0 0x4b3d500 con 0x4a3e160 2015-07-14 16:14:43.516078 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:43.516176 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:43.517877 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 2015-07-14 16:14:43.517921 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 2015-07-14 16:14:43.517942 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set 2015-07-14 16:14:43.517956 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 24 0x4b3c600 mdsbeacon(94247/codetest up:reconnect seq 9 v513) v3 2015-07-14 16:14:43.517996 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:43.518027 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 24 2015-07-14 16:14:43.518047 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:43.518043 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 24 ==== mdsbeacon(94247/codetest up:reconnect seq 9 v513) v3 ==== 120+0+0 (169870696 0 0) 0x4b3c600 con 0x4a3e160 2015-07-14 16:14:43.518096 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 2015-07-14 16:14:47.516088 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 10 v513) v3 -- ?+0 0x4b3d200 con 0x4a3e160 2015-07-14 16:14:47.516161 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:47.516270 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:47.517983 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 2015-07-14 16:14:47.518029 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 2015-07-14 16:14:47.518054 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set 2015-07-14 16:14:47.518068 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 25 0x4b3c300 mdsbeacon(94247/codetest up:reconnect seq 10 v513) v3 2015-07-14 16:14:47.518133 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:47.518163 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 25 2015-07-14 16:14:47.518153 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 25 ==== mdsbeacon(94247/codetest up:reconnect seq 10 v513) v3 ==== 120+0+0 (216387952 0 0) 0x4b3c300 con 0x4a3e160 2015-07-14 16:14:47.518185 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 2015-07-14 16:14:47.518186 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:48.031821 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:48.031859 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_keepalive2 14 2015-07-14 16:14:48.031858 2015-07-14 16:14:48.031909 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:51.516208 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 11 v513) v3 -- ?+0 0x4b3cf00 con 0x4a3e160 2015-07-14 16:14:51.516285 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:51.516385 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:51.518222 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 2015-07-14 16:14:51.518263 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 2015-07-14 16:14:51.518287 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set 2015-07-14 16:14:51.518302 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 26 0x4b3c000 mdsbeacon(94247/codetest up:reconnect seq 11 v513) v3 2015-07-14 16:14:51.518346 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:51.518377 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 26 2015-07-14 16:14:51.518398 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:51.518400 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 26 ==== mdsbeacon(94247/codetest up:reconnect seq 11 v513) v3 ==== 120+0+0 (4060262615 0 0) 0x4b3c000 con 0x4a3e160 2015-07-14 16:14:51.518451 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 2015-07-14 16:14:55.516268 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 12 v513) v3 -- ?+0 0x4b3cc00 con 0x4a3e160 2015-07-14 16:14:55.516344 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:55.516426 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:55.518102 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 2015-07-14 16:14:55.518170 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 2015-07-14 16:14:55.518194 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set 2015-07-14 16:14:55.518209 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 27 0x4b3d800 mdsbeacon(94247/codetest up:reconnect seq 12 v513) v3 2015-07-14 16:14:55.518249 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:55.518280 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 27 2015-07-14 16:14:55.518300 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:55.518294 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 27 ==== mdsbeacon(94247/codetest up:reconnect seq 12 v513) v3 ==== 120+0+0 (24017216 0 0) 0x4b3d800 con 0x4a3e160 2015-07-14 16:14:55.518343 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 2015-07-14 16:14:58.031980 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:58.032017 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_keepalive2 14 2015-07-14 16:14:58.032016 2015-07-14 16:14:58.032089 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:59.516361 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 13 v513) v3 -- ?+0 0x4b3db00 con 0x4a3e160 2015-07-14 16:14:59.516433 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:59.516528 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:59.518354 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 2015-07-14 16:14:59.518395 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 2015-07-14 16:14:59.518417 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set 2015-07-14 16:14:59.518432 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 28 0x4b3d500 mdsbeacon(94247/codetest up:reconnect seq 13 v513) v3 2015-07-14 16:14:59.518470 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:59.518500 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 28 2015-07-14 16:14:59.518521 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:14:59.518522 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 28 ==== mdsbeacon(94247/codetest up:reconnect seq 13 v513) v3 ==== 120+0+0 (4287174887 0 0) 0x4b3d500 con 0x4a3e160 2015-07-14 16:14:59.518570 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 2015-07-14 16:15:03.516471 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 14 v513) v3 -- ?+0 0x4b46f00 con 0x4a3e160 2015-07-14 16:15:03.516539 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:15:03.516642 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:15:03.518299 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 2015-07-14 16:15:03.518348 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 2015-07-14 16:15:03.518372 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set 2015-07-14 16:15:03.518389 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 29 0x4a96000 mdsbeacon(94247/codetest up:reconnect seq 14 v513) v3 2015-07-14 16:15:03.518428 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:15:03.518459 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 29 2015-07-14 16:15:03.518481 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 2015-07-14 16:15:03.518485 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 29 ==== mdsbeacon(94247/codetest up:reconnect seq 14 v513) v3 ==== 120+0+0 (4182559999 0 0) 0x4a96000 con 0x4a3e160 2015-07-14 16:15:03.518537 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 2015-07-14 16:15:05.036630 7f3ea95e8700 1 mds.0.server reconnect gave up on client.4254 172.16.0.60:0/17315 2015-07-14 16:15:05.043112 7f3ea95e8700 -1 ./include/interval_set.h: In function 'void interval_set::insert(T, T) [with T = inodeno_t]' thread 7f3ea95e8700 time 2015-07-14 16:15:05.036693 ./include/interval_set.h: 340: FAILED assert(0) ceph version 9.0.0 (bd7989103911796eb5698cf208b0ccdc3370d707) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x72) [0xa623c2] 2: /usr/bin/ceph-mds() [0x69e82a] 3: (interval_set::insert(interval_set const&)+0x1eb) [0x744e7b] 4: (InoTable::project_release_ids(interval_set&)+0x73) [0x8c3893] 5: (Server::journal_close_session(Session*, int, Context*)+0x168) [0x70d0d8] 6: (Server::kill_session(Session*, Context*)+0x13b) [0x70d69b] 7: (Server::reconnect_tick()+0x128) [0x70d9d8] 8: (MDS::tick()+0x3fc) [0x6b1e0c] 9: (MDSInternalContextBase::complete(int)+0x15b) [0x8e6f2b] 10: (SafeTimer::timer_thread()+0x1af) [0xa5496f] 11: (SafeTimerThread::entry()+0xd) [0xa55f2d] 12: (()+0x6b50) [0x7f3eb0d70b50] 13: (clone()+0x6d) [0x7f3eaf8127bd] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -753> 2015-07-14 16:14:15.016688 7f3eb13e0780 5 asok(0x4a2e000) register_command perfcounters_dump hook 0x4a18050 -752> 2015-07-14 16:14:15.016728 7f3eb13e0780 5 asok(0x4a2e000) register_command 1 hook 0x4a18050 -751> 2015-07-14 16:14:15.016739 7f3eb13e0780 5 asok(0x4a2e000) register_command perf dump hook 0x4a18050 -750> 2015-07-14 16:14:15.016756 7f3eb13e0780 5 asok(0x4a2e000) register_command perfcounters_schema hook 0x4a18050 -749> 2015-07-14 16:14:15.016761 7f3eb13e0780 5 asok(0x4a2e000) register_command 2 hook 0x4a18050 -748> 2015-07-14 16:14:15.016764 7f3eb13e0780 5 asok(0x4a2e000) register_command perf schema hook 0x4a18050 -747> 2015-07-14 16:14:15.016769 7f3eb13e0780 5 asok(0x4a2e000) register_command perf reset hook 0x4a18050 -746> 2015-07-14 16:14:15.016778 7f3eb13e0780 5 asok(0x4a2e000) register_command config show hook 0x4a18050 -745> 2015-07-14 16:14:15.016782 7f3eb13e0780 5 asok(0x4a2e000) register_command config set hook 0x4a18050 -744> 2015-07-14 16:14:15.016786 7f3eb13e0780 5 asok(0x4a2e000) register_command config get hook 0x4a18050 -743> 2015-07-14 16:14:15.016795 7f3eb13e0780 5 asok(0x4a2e000) register_command config diff hook 0x4a18050 -742> 2015-07-14 16:14:15.016799 7f3eb13e0780 5 asok(0x4a2e000) register_command log flush hook 0x4a18050 -741> 2015-07-14 16:14:15.016808 7f3eb13e0780 5 asok(0x4a2e000) register_command log dump hook 0x4a18050 -740> 2015-07-14 16:14:15.016812 7f3eb13e0780 5 asok(0x4a2e000) register_command log reopen hook 0x4a18050 -739> 2015-07-14 16:14:15.019122 7f3eb13e0780 0 ceph version 9.0.0 (bd7989103911796eb5698cf208b0ccdc3370d707), process ceph-mds, pid 25422 -738> 2015-07-14 16:14:15.019452 7f3eb13e0780 10 -- :/0 rank.bind :/0 -737> 2015-07-14 16:14:15.019464 7f3eb13e0780 10 accepter.accepter.bind -736> 2015-07-14 16:14:15.019481 7f3eb13e0780 10 accepter.accepter.bind bound on random port 0.0.0.0:6800/0 -735> 2015-07-14 16:14:15.019491 7f3eb13e0780 10 accepter.accepter.bind bound to 0.0.0.0:6800/0 -734> 2015-07-14 16:14:15.019503 7f3eb13e0780 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6800/25422 need_addr=1 -733> 2015-07-14 16:14:15.020286 7f3eb13e0780 1 finished global_init_daemonize -732> 2015-07-14 16:14:15.020767 7f3eb13e0780 5 asok(0x4a2e000) init /var/run/ceph/ceph-mds.codetest.asok -731> 2015-07-14 16:14:15.020811 7f3eb13e0780 5 asok(0x4a2e000) bind_and_listen /var/run/ceph/ceph-mds.codetest.asok -730> 2015-07-14 16:14:15.021100 7f3eb13e0780 5 asok(0x4a2e000) register_command 0 hook 0x4a16038 -729> 2015-07-14 16:14:15.021124 7f3eb13e0780 5 asok(0x4a2e000) register_command version hook 0x4a16038 -728> 2015-07-14 16:14:15.021131 7f3eb13e0780 5 asok(0x4a2e000) register_command git_version hook 0x4a16038 -727> 2015-07-14 16:14:15.021142 7f3eb13e0780 5 asok(0x4a2e000) register_command help hook 0x4a18030 -726> 2015-07-14 16:14:15.021156 7f3eb13e0780 5 asok(0x4a2e000) register_command get_command_descriptions hook 0x4a180a0 -725> 2015-07-14 16:14:15.021255 7f3eb13e0780 10 monclient(hunting): build_initial_monmap -724> 2015-07-14 16:14:15.021231 7f3eae51a700 5 asok(0x4a2e000) entry start -723> 2015-07-14 16:14:15.030308 7f3eb13e0780 1 -- 0.0.0.0:6800/25422 messenger.start -722> 2015-07-14 16:14:15.030409 7f3eb13e0780 5 adding auth protocol: none -721> 2015-07-14 16:14:15.030424 7f3eb13e0780 5 adding auth protocol: none -720> 2015-07-14 16:14:15.030455 7f3ead6f1700 10 -- 0.0.0.0:6800/25422 reaper_entry start -719> 2015-07-14 16:14:15.030496 7f3ead6f1700 10 -- 0.0.0.0:6800/25422 reaper -718> 2015-07-14 16:14:15.030502 7f3ead6f1700 10 -- 0.0.0.0:6800/25422 reaper done -717> 2015-07-14 16:14:15.030731 7f3eb13e0780 5 asok(0x4a2e000) register_command objecter_requests hook 0x4a180f0 -716> 2015-07-14 16:14:15.030799 7f3eb13e0780 10 -- 0.0.0.0:6800/25422 ready 0.0.0.0:6800/25422 -715> 2015-07-14 16:14:15.030906 7f3eb13e0780 1 accepter.accepter.start -714> 2015-07-14 16:14:15.030954 7f3eb13e0780 10 monclient(hunting): init -713> 2015-07-14 16:14:15.030978 7f3eb13e0780 5 adding auth protocol: none -712> 2015-07-14 16:14:15.030991 7f3eb13e0780 10 monclient(hunting): auth_supported 1 method none -711> 2015-07-14 16:14:15.031016 7f3eab6ed700 10 accepter.accepter starting -710> 2015-07-14 16:14:15.031190 7f3eb13e0780 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -709> 2015-07-14 16:14:15.031213 7f3eb13e0780 -1 mds.-1.0 log_to_monitors {default=true} -708> 2015-07-14 16:14:15.031243 7f3eb13e0780 10 monclient(hunting): _reopen_session rank -1 name -707> 2015-07-14 16:14:15.031252 7f3eb13e0780 10 -- 0.0.0.0:6800/25422 connect_rank to 172.16.5.170:6789/0, creating pipe and registering -706> 2015-07-14 16:14:15.031344 7f3eb13e0780 10 -- 0.0.0.0:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e160).register_pipe -705> 2015-07-14 16:14:15.031395 7f3eb13e0780 10 -- 0.0.0.0:6800/25422 get_connection mon.2 172.16.5.170:6789/0 new 0x4a7e000 -704> 2015-07-14 16:14:15.031396 7f3eb13d8700 10 -- 0.0.0.0:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e160).writer: state = connecting policy.server=0 -703> 2015-07-14 16:14:15.031433 7f3eb13d8700 10 -- 0.0.0.0:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e160).connect 0 -702> 2015-07-14 16:14:15.031451 7f3eb13d8700 10 -- 0.0.0.0:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e160).connecting to 172.16.5.170:6789/0 -701> 2015-07-14 16:14:15.031455 7f3eb13e0780 10 monclient(hunting): picked mon.noname-c con 0x4a3e160 addr 172.16.5.170:6789/0 -700> 2015-07-14 16:14:15.031510 7f3eb13e0780 10 monclient(hunting): _send_mon_message to mon.noname-c at 172.16.5.170:6789/0 -699> 2015-07-14 16:14:15.031564 7f3eb13e0780 1 -- 0.0.0.0:6800/25422 --> 172.16.5.170:6789/0 -- auth(proto 0 33 bytes epoch 0) v1 -- ?+0 0x4a2c240 con 0x4a3e160 -698> 2015-07-14 16:14:15.031600 7f3eb13e0780 10 monclient(hunting): renew_subs -697> 2015-07-14 16:14:15.032184 7f3eb13d8700 1 -- 172.16.0.50:6800/25422 learned my addr 172.16.0.50:6800/25422 -696> 2015-07-14 16:14:15.032240 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=1 pgs=0 cs=0 l=1 c=0x4a3e160).connect sent my addr 172.16.0.50:6800/25422 -695> 2015-07-14 16:14:15.032263 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=1 pgs=0 cs=0 l=1 c=0x4a3e160).connect sending gseq=1 cseq=0 proto=15 -694> 2015-07-14 16:14:15.032520 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).connect success 1, lossy = 1, features 1125899906842623 -693> 2015-07-14 16:14:15.032626 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -692> 2015-07-14 16:14:15.032657 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_keepalive2 14 2015-07-14 16:14:15.032653 -691> 2015-07-14 16:14:15.032769 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -690> 2015-07-14 16:14:15.033222 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 495 from dispatch throttler 0/104857600 -689> 2015-07-14 16:14:15.033271 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -688> 2015-07-14 16:14:15.033313 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -687> 2015-07-14 16:14:15.033334 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 1 0x4a2c6c0 mon_map magic: 0 v1 -686> 2015-07-14 16:14:15.033363 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 24 from dispatch throttler 495/104857600 -685> 2015-07-14 16:14:15.033380 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -684> 2015-07-14 16:14:15.033399 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -683> 2015-07-14 16:14:15.033367 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -682> 2015-07-14 16:14:15.033418 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 1 -681> 2015-07-14 16:14:15.033428 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 2 0x4a2cb40 auth_reply(proto 1 0 (0) Success) v1 -680> 2015-07-14 16:14:15.033423 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 1 ==== mon_map magic: 0 v1 ==== 495+0+0 (2629194327 0 0) 0x4a2c6c0 con 0x4a3e160 -679> 2015-07-14 16:14:15.033483 7f3eac6ef700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 -678> 2015-07-14 16:14:15.033473 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -677> 2015-07-14 16:14:15.033491 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 2 -676> 2015-07-14 16:14:15.033503 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -675> 2015-07-14 16:14:15.033510 7f3eac6ef700 10 monclient(hunting): got monmap 2, mon.noname-c is now rank -1 -674> 2015-07-14 16:14:15.033515 7f3eac6ef700 10 monclient(hunting): dump: epoch 2 fsid a09e3308-e4ed-4c15-973e-d21142b6be04 last_changed 2015-05-28 18:29:46.831963 created 0.000000 0: 172.16.0.50:6789/0 mon.codetest 1: 172.16.0.60:6789/0 mon.cctext 2: 172.16.5.170:6789/0 mon.cloudCode04 -673> 2015-07-14 16:14:15.033603 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 495 to dispatch throttler 519/104857600 -672> 2015-07-14 16:14:15.033627 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 2 ==== auth_reply(proto 1 0 (0) Success) v1 ==== 24+0+0 (763678429 0 0) 0x4a2cb40 con 0x4a3e160 -671> 2015-07-14 16:14:15.033667 7f3eac6ef700 10 monclient(hunting): my global_id is 94247 -670> 2015-07-14 16:14:15.033673 7f3eac6ef700 1 monclient(hunting): found mon.cloudCode04 -669> 2015-07-14 16:14:15.033676 7f3eac6ef700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -668> 2015-07-14 16:14:15.033683 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mon_subscribe({monmap=0+}) v2 -- ?+0 0x4a38400 con 0x4a3e160 -667> 2015-07-14 16:14:15.033737 7f3eac6ef700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2015-07-14 16:13:45.033734) -666> 2015-07-14 16:14:15.033740 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -665> 2015-07-14 16:14:15.033761 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 24 to dispatch throttler 24/104857600 -664> 2015-07-14 16:14:15.033771 7f3eb13e0780 5 monclient: authenticate success, global_id 94247 -663> 2015-07-14 16:14:15.033803 7f3eb13e0780 10 monclient: renew_subs -662> 2015-07-14 16:14:15.033802 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -661> 2015-07-14 16:14:15.033811 7f3eb13e0780 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -660> 2015-07-14 16:14:15.033824 7f3eb13e0780 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mon_subscribe({monmap=3+,osdmap=0}) v2 -- ?+0 0x4a38200 con 0x4a3e160 -659> 2015-07-14 16:14:15.033848 7f3eb13e0780 10 monclient: renew_subs -658> 2015-07-14 16:14:15.033856 7f3eb13e0780 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -657> 2015-07-14 16:14:15.033848 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -656> 2015-07-14 16:14:15.033871 7f3eb13e0780 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mon_subscribe({mdsmap=0+,monmap=3+,osdmap=0}) v2 -- ?+0 0x4a38c00 con 0x4a3e160 -655> 2015-07-14 16:14:15.033912 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -654> 2015-07-14 16:14:15.033958 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -653> 2015-07-14 16:14:15.034411 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 495 from dispatch throttler 0/104857600 -652> 2015-07-14 16:14:15.034439 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -651> 2015-07-14 16:14:15.034454 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -650> 2015-07-14 16:14:15.034480 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 3 0x4a2c900 mon_map magic: 0 v1 -649> 2015-07-14 16:14:15.034535 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 20 from dispatch throttler 495/104857600 -648> 2015-07-14 16:14:15.034559 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -647> 2015-07-14 16:14:15.034541 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -646> 2015-07-14 16:14:15.034571 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 3 -645> 2015-07-14 16:14:15.034595 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -644> 2015-07-14 16:14:15.034594 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -643> 2015-07-14 16:14:15.034584 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 495+0+0 (2629194327 0 0) 0x4a2c900 con 0x4a3e160 -642> 2015-07-14 16:14:15.034618 7f3eac6ef700 10 monclient: handle_monmap mon_map magic: 0 v1 -641> 2015-07-14 16:14:15.034616 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 4 0x4a8e000 mon_subscribe_ack(300s) v1 -640> 2015-07-14 16:14:15.034641 7f3eac6ef700 10 monclient: got monmap 2, mon.cloudCode04 is now rank 2 -639> 2015-07-14 16:14:15.034646 7f3eac6ef700 10 monclient: dump: epoch 2 fsid a09e3308-e4ed-4c15-973e-d21142b6be04 last_changed 2015-05-28 18:29:46.831963 created 0.000000 0: 172.16.0.50:6789/0 mon.codetest 1: 172.16.0.60:6789/0 mon.cctext 2: 172.16.5.170:6789/0 mon.cloudCode04 -638> 2015-07-14 16:14:15.034684 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -637> 2015-07-14 16:14:15.034692 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 495 to dispatch throttler 515/104857600 -636> 2015-07-14 16:14:15.034688 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 8077 from dispatch throttler 515/104857600 -635> 2015-07-14 16:14:15.034699 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 4 -634> 2015-07-14 16:14:15.034709 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 4 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (892755957 0 0) 0x4a8e000 con 0x4a3e160 -633> 2015-07-14 16:14:15.034712 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -632> 2015-07-14 16:14:15.034734 7f3eac6ef700 10 monclient: handle_subscribe_ack sent 2015-07-14 16:14:15.031616 renew after 2015-07-14 16:16:45.031616 -631> 2015-07-14 16:14:15.034751 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 20 to dispatch throttler 8097/104857600 -630> 2015-07-14 16:14:15.034795 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -629> 2015-07-14 16:14:15.034855 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -628> 2015-07-14 16:14:15.034876 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 5 0x4a2cd80 osd_map(2382..2382 src has 1757..2382) v3 -627> 2015-07-14 16:14:15.034901 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 20 from dispatch throttler 8077/104857600 -626> 2015-07-14 16:14:15.034906 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -625> 2015-07-14 16:14:15.034918 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -624> 2015-07-14 16:14:15.034920 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 5 -623> 2015-07-14 16:14:15.034912 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 5 ==== osd_map(2382..2382 src has 1757..2382) v3 ==== 8077+0+0 (2554887417 0 0) 0x4a2cd80 con 0x4a3e160 -622> 2015-07-14 16:14:15.034932 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -621> 2015-07-14 16:14:15.034933 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -620> 2015-07-14 16:14:15.034962 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 6 0x4a8e3c0 mon_subscribe_ack(300s) v1 -619> 2015-07-14 16:14:15.034984 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 680 from dispatch throttler 8097/104857600 -618> 2015-07-14 16:14:15.034989 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -617> 2015-07-14 16:14:15.034999 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -616> 2015-07-14 16:14:15.035003 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 6 -615> 2015-07-14 16:14:15.035013 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -614> 2015-07-14 16:14:15.035014 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -613> 2015-07-14 16:14:15.035039 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 7 0x4a2cb40 mdsmap(e 510) v1 -612> 2015-07-14 16:14:15.035062 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -611> 2015-07-14 16:14:15.035058 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 8077 from dispatch throttler 8777/104857600 -610> 2015-07-14 16:14:15.035074 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 7 -609> 2015-07-14 16:14:15.035087 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -608> 2015-07-14 16:14:15.035106 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -607> 2015-07-14 16:14:15.035139 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -606> 2015-07-14 16:14:15.035155 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 8 0x4a2c6c0 osd_map(2382..2382 src has 1757..2382) v3 -605> 2015-07-14 16:14:15.035175 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 20 from dispatch throttler 16854/104857600 -604> 2015-07-14 16:14:15.035179 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -603> 2015-07-14 16:14:15.035188 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -602> 2015-07-14 16:14:15.035193 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 8 -601> 2015-07-14 16:14:15.035200 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -600> 2015-07-14 16:14:15.035205 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -599> 2015-07-14 16:14:15.035232 7f3eac6ef700 10 monclient: renew_subs -598> 2015-07-14 16:14:15.035228 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 9 0x4a8e1e0 mon_subscribe_ack(300s) v1 -597> 2015-07-14 16:14:15.035245 7f3eac6ef700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -596> 2015-07-14 16:14:15.035254 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -595> 2015-07-14 16:14:15.035255 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mon_subscribe({mdsmap=0+,monmap=3+,osdmap=2383}) v2 -- ?+0 0x4a38200 con 0x4a3e160 -594> 2015-07-14 16:14:15.035268 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 9 -593> 2015-07-14 16:14:15.035295 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 8077 to dispatch throttler 16874/104857600 -592> 2015-07-14 16:14:15.035309 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 6 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (892755957 0 0) 0x4a8e3c0 con 0x4a3e160 -591> 2015-07-14 16:14:15.035313 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -590> 2015-07-14 16:14:15.035325 7f3eac6ef700 10 monclient: handle_subscribe_ack sent 2015-07-14 16:14:15.035242 renew after 2015-07-14 16:16:45.035242 -589> 2015-07-14 16:14:15.035349 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 20 to dispatch throttler 8797/104857600 -588> 2015-07-14 16:14:15.035365 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 7 ==== mdsmap(e 510) v1 ==== 680+0+0 (2892432026 0 0) 0x4a2cb40 con 0x4a3e160 -587> 2015-07-14 16:14:15.035479 7f3eb13e0780 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -586> 2015-07-14 16:14:15.035498 7f3eb13e0780 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:boot seq 1 v0) v3 -- ?+0 0x4a96000 con 0x4a3e160 -585> 2015-07-14 16:14:15.035608 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -584> 2015-07-14 16:14:15.035666 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -583> 2015-07-14 16:14:15.035745 7f3eb13e0780 5 asok(0x4a2e000) register_command status hook 0x4a181a0 -582> 2015-07-14 16:14:15.035767 7f3eb13e0780 5 asok(0x4a2e000) register_command dump_ops_in_flight hook 0x4a181a0 -581> 2015-07-14 16:14:15.035777 7f3eb13e0780 5 asok(0x4a2e000) register_command ops hook 0x4a181a0 -580> 2015-07-14 16:14:15.035785 7f3eb13e0780 5 asok(0x4a2e000) register_command dump_historic_ops hook 0x4a181a0 -579> 2015-07-14 16:14:15.035795 7f3eb13e0780 5 asok(0x4a2e000) register_command scrub_path hook 0x4a181a0 -578> 2015-07-14 16:14:15.035805 7f3eb13e0780 5 asok(0x4a2e000) register_command flush_path hook 0x4a181a0 -577> 2015-07-14 16:14:15.035814 7f3eb13e0780 5 asok(0x4a2e000) register_command export dir hook 0x4a181a0 -576> 2015-07-14 16:14:15.035824 7f3eb13e0780 5 asok(0x4a2e000) register_command dump cache hook 0x4a181a0 -575> 2015-07-14 16:14:15.035832 7f3eb13e0780 5 asok(0x4a2e000) register_command session evict hook 0x4a181a0 -574> 2015-07-14 16:14:15.035841 7f3eb13e0780 5 asok(0x4a2e000) register_command osdmap barrier hook 0x4a181a0 -573> 2015-07-14 16:14:15.035847 7f3eb13e0780 5 asok(0x4a2e000) register_command session ls hook 0x4a181a0 -572> 2015-07-14 16:14:15.035868 7f3eb13e0780 5 asok(0x4a2e000) register_command flush journal hook 0x4a181a0 -571> 2015-07-14 16:14:15.035878 7f3eb13e0780 5 asok(0x4a2e000) register_command force_readonly hook 0x4a181a0 -570> 2015-07-14 16:14:15.035895 7f3eb13e0780 5 asok(0x4a2e000) register_command get subtrees hook 0x4a181a0 -569> 2015-07-14 16:14:15.035948 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 680 from dispatch throttler 8777/104857600 -568> 2015-07-14 16:14:15.035980 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -567> 2015-07-14 16:14:15.035974 7f3eac6ef700 5 mds.-1.0 handle_mds_map epoch 510 from mon.2 -566> 2015-07-14 16:14:15.036007 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -565> 2015-07-14 16:14:15.036010 7f3eb13e0780 10 -- 172.16.0.50:6800/25422 wait: waiting for dispatch queue -564> 2015-07-14 16:14:15.036021 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 10 0x4a2d680 mdsmap(e 510) v1 -563> 2015-07-14 16:14:15.036038 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 20 from dispatch throttler 9457/104857600 -562> 2015-07-14 16:14:15.036070 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -561> 2015-07-14 16:14:15.036062 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -560> 2015-07-14 16:14:15.036085 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -559> 2015-07-14 16:14:15.036093 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 10 -558> 2015-07-14 16:14:15.036096 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 11 0x4a8e5a0 mon_subscribe_ack(300s) v1 -557> 2015-07-14 16:14:15.036103 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 680 to dispatch throttler 9477/104857600 -556> 2015-07-14 16:14:15.036113 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -555> 2015-07-14 16:14:15.036131 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 11 -554> 2015-07-14 16:14:15.036143 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -553> 2015-07-14 16:14:15.036130 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 8 ==== osd_map(2382..2382 src has 1757..2382) v3 ==== 8077+0+0 (2554887417 0 0) 0x4a2c6c0 con 0x4a3e160 -552> 2015-07-14 16:14:15.036194 7f3eac6ef700 10 monclient: renew_subs -551> 2015-07-14 16:14:15.036202 7f3eac6ef700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -550> 2015-07-14 16:14:15.036209 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mon_subscribe({mdsmap=511+,monmap=3+,osdmap=2383}) v2 -- ?+0 0x4a38600 con 0x4a3e160 -549> 2015-07-14 16:14:15.036232 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 8077 to dispatch throttler 8797/104857600 -548> 2015-07-14 16:14:15.036237 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -547> 2015-07-14 16:14:15.036242 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 9 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (892755957 0 0) 0x4a8e1e0 con 0x4a3e160 -546> 2015-07-14 16:14:15.036290 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -545> 2015-07-14 16:14:15.036258 7f3eac6ef700 10 monclient: handle_subscribe_ack sent 2015-07-14 16:14:15.036199 renew after 2015-07-14 16:16:45.036199 -544> 2015-07-14 16:14:15.036317 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 20 to dispatch throttler 720/104857600 -543> 2015-07-14 16:14:15.036330 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 10 ==== mdsmap(e 510) v1 ==== 680+0+0 (2892432026 0 0) 0x4a2d680 con 0x4a3e160 -542> 2015-07-14 16:14:15.036347 7f3eac6ef700 5 mds.-1.-1 handle_mds_map epoch 510 from mon.2 -541> 2015-07-14 16:14:15.036352 7f3eac6ef700 5 mds.-1.-1 old map epoch 510 <= 510, discarding -540> 2015-07-14 16:14:15.036357 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 680 to dispatch throttler 700/104857600 -539> 2015-07-14 16:14:15.036370 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 11 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (892755957 0 0) 0x4a8e5a0 con 0x4a3e160 -538> 2015-07-14 16:14:15.036395 7f3eac6ef700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring -537> 2015-07-14 16:14:15.036400 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 20 to dispatch throttler 20/104857600 -536> 2015-07-14 16:14:15.036849 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 20 from dispatch throttler 0/104857600 -535> 2015-07-14 16:14:15.036881 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -534> 2015-07-14 16:14:15.036898 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -533> 2015-07-14 16:14:15.036911 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 12 0x4a8e3c0 mon_subscribe_ack(300s) v1 -532> 2015-07-14 16:14:15.036948 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 12 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (892755957 0 0) 0x4a8e3c0 con 0x4a3e160 -531> 2015-07-14 16:14:15.036948 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -530> 2015-07-14 16:14:15.036976 7f3eac6ef700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring -529> 2015-07-14 16:14:15.036981 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 20 to dispatch throttler 20/104857600 -528> 2015-07-14 16:14:15.036979 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 12 -527> 2015-07-14 16:14:15.037001 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -526> 2015-07-14 16:14:15.170211 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 358 from dispatch throttler 0/104857600 -525> 2015-07-14 16:14:15.170253 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -524> 2015-07-14 16:14:15.170274 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -523> 2015-07-14 16:14:15.170312 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 13 0x4a2d440 osd_map(2383..2383 src has 1757..2383) v3 -522> 2015-07-14 16:14:15.170354 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -521> 2015-07-14 16:14:15.170384 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 13 -520> 2015-07-14 16:14:15.170402 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -519> 2015-07-14 16:14:15.170400 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 13 ==== osd_map(2383..2383 src has 1757..2383) v3 ==== 358+0+0 (3081206777 0 0) 0x4a2d440 con 0x4a3e160 -518> 2015-07-14 16:14:15.170486 7f3eac6ef700 10 monclient: renew_subs -517> 2015-07-14 16:14:15.170502 7f3eac6ef700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -516> 2015-07-14 16:14:15.170521 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mon_subscribe({mdsmap=511+,monmap=3+,osdmap=2384}) v2 -- ?+0 0x4a38600 con 0x4a3e160 -515> 2015-07-14 16:14:15.170557 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 358 to dispatch throttler 358/104857600 -514> 2015-07-14 16:14:15.170593 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -513> 2015-07-14 16:14:15.170671 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -512> 2015-07-14 16:14:15.236983 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 20 from dispatch throttler 0/104857600 -511> 2015-07-14 16:14:15.237025 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -510> 2015-07-14 16:14:15.237042 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -509> 2015-07-14 16:14:15.237057 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 14 0x4a8e000 mon_subscribe_ack(300s) v1 -508> 2015-07-14 16:14:15.237100 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -507> 2015-07-14 16:14:15.237130 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 14 -506> 2015-07-14 16:14:15.237151 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -505> 2015-07-14 16:14:15.237145 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 14 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (892755957 0 0) 0x4a8e000 con 0x4a3e160 -504> 2015-07-14 16:14:15.237182 7f3eac6ef700 10 monclient: handle_subscribe_ack sent 2015-07-14 16:14:15.170498 renew after 2015-07-14 16:16:45.170498 -503> 2015-07-14 16:14:15.237217 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 20 to dispatch throttler 20/104857600 -502> 2015-07-14 16:14:15.396583 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 672 from dispatch throttler 0/104857600 -501> 2015-07-14 16:14:15.396650 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -500> 2015-07-14 16:14:15.396671 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -499> 2015-07-14 16:14:15.396686 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 15 0x4a2d200 mdsmap(e 511) v1 -498> 2015-07-14 16:14:15.396731 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -497> 2015-07-14 16:14:15.396762 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 15 -496> 2015-07-14 16:14:15.396765 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 15 ==== mdsmap(e 511) v1 ==== 672+0+0 (2417860917 0 0) 0x4a2d200 con 0x4a3e160 -495> 2015-07-14 16:14:15.396780 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -494> 2015-07-14 16:14:15.396798 7f3eac6ef700 5 mds.-1.-1 handle_mds_map epoch 511 from mon.2 -493> 2015-07-14 16:14:15.396841 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 mark_down 172.16.0.50:6800/25308 -- pipe dne -492> 2015-07-14 16:14:15.396876 7f3eac6ef700 1 mds.-1.0 handle_mds_map standby -491> 2015-07-14 16:14:15.396918 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 672 to dispatch throttler 672/104857600 -490> 2015-07-14 16:14:15.496974 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 680 from dispatch throttler 0/104857600 -489> 2015-07-14 16:14:15.497044 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -488> 2015-07-14 16:14:15.497063 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -487> 2015-07-14 16:14:15.497078 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 16 0x4a2cfc0 mdsmap(e 512) v1 -486> 2015-07-14 16:14:15.497120 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -485> 2015-07-14 16:14:15.497150 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 16 -484> 2015-07-14 16:14:15.497171 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -483> 2015-07-14 16:14:15.497172 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 16 ==== mdsmap(e 512) v1 ==== 680+0+0 (1254507515 0 0) 0x4a2cfc0 con 0x4a3e160 -482> 2015-07-14 16:14:15.497207 7f3eac6ef700 5 mds.-1.0 handle_mds_map epoch 512 from mon.2 -481> 2015-07-14 16:14:15.497251 7f3eac6ef700 1 mds.0.80 handle_mds_map i am now mds.0.80 -480> 2015-07-14 16:14:15.497265 7f3eac6ef700 1 mds.0.80 handle_mds_map state change up:standby --> up:replay -479> 2015-07-14 16:14:15.497281 7f3eac6ef700 1 mds.0.80 replay_start -478> 2015-07-14 16:14:15.497294 7f3eac6ef700 1 mds.0.80 recovery set is -477> 2015-07-14 16:14:15.497305 7f3eac6ef700 2 mds.0.80 boot_start 0: opening inotable -476> 2015-07-14 16:14:15.497430 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 connect_rank to 172.16.5.230:6808/27037, creating pipe and registering -475> 2015-07-14 16:14:15.497538 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e420).register_pipe -474> 2015-07-14 16:14:15.497572 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 get_connection osd.5 172.16.5.230:6808/27037 new 0x4a83000 -473> 2015-07-14 16:14:15.497581 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e420).writer: state = connecting policy.server=0 -472> 2015-07-14 16:14:15.497624 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e420).connect 0 -471> 2015-07-14 16:14:15.497654 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e420).connecting to 172.16.5.230:6808/27037 -470> 2015-07-14 16:14:15.497687 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.230:6808/27037 -- osd_op(mds.0.80:1 mds0_inotable [read 0~0] 1.b852b893 ack+read+known_if_redirected e2383) v5 -- ?+0 0x4a5af00 con 0x4a3e420 -469> 2015-07-14 16:14:15.497726 7f3eac6ef700 2 mds.0.80 boot_start 0: opening sessionmap -468> 2015-07-14 16:14:15.497784 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 connect_rank to 172.16.0.50:6801/15304, creating pipe and registering -467> 2015-07-14 16:14:15.497882 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e6e0).register_pipe -466> 2015-07-14 16:14:15.497907 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 get_connection osd.0 172.16.0.50:6801/15304 new 0x4ab5000 -465> 2015-07-14 16:14:15.497924 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 --> 172.16.0.50:6801/15304 -- osd_op(mds.0.80:2 mds0_sessionmap [omap-get-header 0~0,omap-get-vals 0~16] 1.3270c60b ack+read+known_if_redirected e2383) v5 -- ?+0 0x4a5b400 con 0x4a3e6e0 -464> 2015-07-14 16:14:15.497923 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e6e0).writer: state = connecting policy.server=0 -463> 2015-07-14 16:14:15.497982 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e6e0).connect 0 -462> 2015-07-14 16:14:15.498011 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e6e0).connecting to 172.16.0.50:6801/15304 -461> 2015-07-14 16:14:15.498024 7f3eac6ef700 2 mds.0.80 boot_start 0: opening mds log -460> 2015-07-14 16:14:15.498033 7f3eac6ef700 5 mds.0.log open discovering log bounds -459> 2015-07-14 16:14:15.498122 7f3eac6ef700 2 mds.0.80 boot_start 0: opening snap table -458> 2015-07-14 16:14:15.498155 7f3ea73e2700 4 mds.0.journalpointer Reading journal pointer '400.00000000' -457> 2015-07-14 16:14:15.498161 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 connect_rank to 172.16.5.170:6808/9971, creating pipe and registering -456> 2015-07-14 16:14:15.498248 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e580).register_pipe -455> 2015-07-14 16:14:15.498273 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 get_connection osd.7 172.16.5.170:6808/9971 new 0x4ab0000 -454> 2015-07-14 16:14:15.498257 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e580).writer: state = connecting policy.server=0 -453> 2015-07-14 16:14:15.498293 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e580).connect 0 -452> 2015-07-14 16:14:15.498319 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e580).connecting to 172.16.5.170:6808/9971 -451> 2015-07-14 16:14:15.498325 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6808/9971 -- osd_op(mds.0.80:3 mds_snaptable [read 0~0] 1.d90270ad ack+read+known_if_redirected e2383) v5 -- ?+0 0x4a5b180 con 0x4a3e580 -450> 2015-07-14 16:14:15.498295 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=1 pgs=0 cs=0 l=1 c=0x4a3e420).connect sent my addr 172.16.0.50:6800/25422 -449> 2015-07-14 16:14:15.498356 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=1 pgs=0 cs=0 l=1 c=0x4a3e420).connect.authorizer_len=25 protocol=1 -448> 2015-07-14 16:14:15.498368 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=1 pgs=0 cs=0 l=1 c=0x4a3e420).connect sending gseq=2 cseq=0 proto=24 -447> 2015-07-14 16:14:15.498378 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 680 to dispatch throttler 680/104857600 -446> 2015-07-14 16:14:15.498390 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=1 pgs=0 cs=0 l=1 c=0x4a3e6e0).connect sent my addr 172.16.0.50:6800/25422 -445> 2015-07-14 16:14:15.498444 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=1 pgs=0 cs=0 l=1 c=0x4a3e6e0).connect.authorizer_len=25 protocol=1 -444> 2015-07-14 16:14:15.498456 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=1 pgs=0 cs=0 l=1 c=0x4a3e6e0).connect sending gseq=3 cseq=0 proto=24 -443> 2015-07-14 16:14:15.498423 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.230:6808/27037 -- osd_op(mds.0.80:4 400.00000000 [read 0~0] 1.64e96f8f ack+read+known_if_redirected e2383) v5 -- ?+0 0x4a5b680 con 0x4a3e420 -442> 2015-07-14 16:14:15.498657 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).connect success 1, lossy = 1, features 1125899906842623 -441> 2015-07-14 16:14:15.498734 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).connect success 1, lossy = 1, features 1125899906842623 -440> 2015-07-14 16:14:15.498783 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).writer: state = open policy.server=0 -439> 2015-07-14 16:14:15.498890 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -438> 2015-07-14 16:14:15.498903 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).writer: state = open policy.server=0 -437> 2015-07-14 16:14:15.498959 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -436> 2015-07-14 16:14:15.498995 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -435> 2015-07-14 16:14:15.499020 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=1 pgs=0 cs=0 l=1 c=0x4a3e580).connect sent my addr 172.16.0.50:6800/25422 -434> 2015-07-14 16:14:15.499062 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=1 pgs=0 cs=0 l=1 c=0x4a3e580).connect.authorizer_len=25 protocol=1 -433> 2015-07-14 16:14:15.499073 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=1 pgs=0 cs=0 l=1 c=0x4a3e580).connect sending gseq=4 cseq=0 proto=24 -432> 2015-07-14 16:14:15.499442 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).connect success 1, lossy = 1, features 1125899906842623 -431> 2015-07-14 16:14:15.499565 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -430> 2015-07-14 16:14:15.499620 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -429> 2015-07-14 16:14:15.500047 7f3ea62df700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).reader wants 2355 from dispatch throttler 0/104857600 -428> 2015-07-14 16:14:15.500121 7f3ea62df700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).aborted = 0 -427> 2015-07-14 16:14:15.500167 7f3ea62df700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).reader got message 1 0x4a76000 osd_op_reply(2 mds0_sessionmap [omap-get-header 0~0,omap-get-vals 0~16] v0'0 uv3088 ondisk = 0) v6 -426> 2015-07-14 16:14:15.500212 7f3ea62df700 1 -- 172.16.0.50:6800/25422 <== osd.0 172.16.0.50:6801/15304 1 ==== osd_op_reply(2 mds0_sessionmap [omap-get-header 0~0,omap-get-vals 0~16] v0'0 uv3088 ondisk = 0) v6 ==== 224+0+2131 (2210788316 0 3413253222) 0x4a76000 con 0x4a3e6e0 -425> 2015-07-14 16:14:15.500223 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).writer: state = open policy.server=0 -424> 2015-07-14 16:14:15.500245 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).write_ack 1 -423> 2015-07-14 16:14:15.500264 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).writer: state = open policy.server=0 -422> 2015-07-14 16:14:15.500283 7f3ea62df700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 2355 to dispatch throttler 2355/104857600 -421> 2015-07-14 16:14:15.501683 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader wants 214 from dispatch throttler 0/104857600 -420> 2015-07-14 16:14:15.501752 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).aborted = 0 -419> 2015-07-14 16:14:15.501790 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader got message 1 0x4a762c0 osd_op_reply(1 mds0_inotable [read 0~34] v0'0 uv5438 ondisk = 0) v6 -418> 2015-07-14 16:14:15.501810 7f3ea61de700 1 -- 172.16.0.50:6800/25422 <== osd.5 172.16.5.230:6808/27037 1 ==== osd_op_reply(1 mds0_inotable [read 0~34] v0'0 uv5438 ondisk = 0) v6 ==== 180+0+34 (3141072272 0 1695280775) 0x4a762c0 con 0x4a3e420 -417> 2015-07-14 16:14:15.501855 7f3ea61de700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 214 to dispatch throttler 214/104857600 -416> 2015-07-14 16:14:15.501836 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -415> 2015-07-14 16:14:15.501878 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).write_ack 1 -414> 2015-07-14 16:14:15.501900 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -413> 2015-07-14 16:14:15.502098 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader wants 201 from dispatch throttler 0/104857600 -412> 2015-07-14 16:14:15.502127 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).aborted = 0 -411> 2015-07-14 16:14:15.502101 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).reader wants 226 from dispatch throttler 0/104857600 -410> 2015-07-14 16:14:15.502152 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).aborted = 0 -409> 2015-07-14 16:14:15.502153 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader got message 2 0x4a762c0 osd_op_reply(4 400.00000000 [read 0~22] v0'0 uv6079 ondisk = 0) v6 -408> 2015-07-14 16:14:15.502177 7f3ea61de700 1 -- 172.16.0.50:6800/25422 <== osd.5 172.16.5.230:6808/27037 2 ==== osd_op_reply(4 400.00000000 [read 0~22] v0'0 uv6079 ondisk = 0) v6 ==== 179+0+22 (3631818520 0 3825830296) 0x4a762c0 con 0x4a3e420 -407> 2015-07-14 16:14:15.502187 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).reader got message 1 0x4a76b00 osd_op_reply(3 mds_snaptable [read 0~46] v0'0 uv5293 ondisk = 0) v6 -406> 2015-07-14 16:14:15.502187 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -405> 2015-07-14 16:14:15.502208 7f3ea61de700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 201 to dispatch throttler 427/104857600 -404> 2015-07-14 16:14:15.502207 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).write_ack 2 -403> 2015-07-14 16:14:15.502206 7f3ea60dd700 1 -- 172.16.0.50:6800/25422 <== osd.7 172.16.5.170:6808/9971 1 ==== osd_op_reply(3 mds_snaptable [read 0~46] v0'0 uv5293 ondisk = 0) v6 ==== 180+0+46 (1566169833 0 2603725198) 0x4a76b00 con 0x4a3e580 -402> 2015-07-14 16:14:15.502222 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -401> 2015-07-14 16:14:15.502247 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 226 to dispatch throttler 226/104857600 -400> 2015-07-14 16:14:15.502230 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -399> 2015-07-14 16:14:15.502281 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).write_ack 1 -398> 2015-07-14 16:14:15.502306 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -397> 2015-07-14 16:14:15.502307 7f3ea73e2700 1 mds.0.journaler(ro) recover start -396> 2015-07-14 16:14:15.502319 7f3ea73e2700 1 mds.0.journaler(ro) read_head -395> 2015-07-14 16:14:15.502376 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.230:6808/27037 -- osd_op(mds.0.80:5 200.00000000 [read 0~0] 1.844f3494 ack+read+known_if_redirected e2383) v5 -- ?+0 0x4a5b680 con 0x4a3e420 -394> 2015-07-14 16:14:15.502425 7f3ea73e2700 4 mds.0.log Waiting for journal 200 to recover... -393> 2015-07-14 16:14:15.502430 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -392> 2015-07-14 16:14:15.502484 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -391> 2015-07-14 16:14:15.503277 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader wants 269 from dispatch throttler 0/104857600 -390> 2015-07-14 16:14:15.503310 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).aborted = 0 -389> 2015-07-14 16:14:15.503336 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader got message 3 0x4a762c0 osd_op_reply(5 200.00000000 [read 0~90] v0'0 uv9056 ondisk = 0) v6 -388> 2015-07-14 16:14:15.503351 7f3ea61de700 1 -- 172.16.0.50:6800/25422 <== osd.5 172.16.5.230:6808/27037 3 ==== osd_op_reply(5 200.00000000 [read 0~90] v0'0 uv9056 ondisk = 0) v6 ==== 179+0+90 (1153837370 0 900091797) 0x4a762c0 con 0x4a3e420 -387> 2015-07-14 16:14:15.503386 7f3ea61de700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 269 to dispatch throttler 269/104857600 -386> 2015-07-14 16:14:15.503376 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -385> 2015-07-14 16:14:15.503412 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).write_ack 3 -384> 2015-07-14 16:14:15.503433 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -383> 2015-07-14 16:14:15.503426 7f3ea9eea700 1 mds.0.journaler(ro) _finish_read_head loghead(trim 4194304, expire 4194304, write 4221582, stream_format 1). probing for end of log (from 4221582)... -382> 2015-07-14 16:14:15.503448 7f3ea9eea700 1 mds.0.journaler(ro) probing for end of the log -381> 2015-07-14 16:14:15.503583 7f3ea9eea700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.230:6808/27037 -- osd_op(mds.0.80:6 200.00000001 [stat] 1.6e5f474 ack+read+rwordered+known_if_redirected e2383) v5 -- ?+0 0x4a5b900 con 0x4a3e420 -380> 2015-07-14 16:14:15.503648 7f3ea9eea700 10 -- 172.16.0.50:6800/25422 connect_rank to 172.16.5.222:6804/13376, creating pipe and registering -379> 2015-07-14 16:14:15.503639 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -378> 2015-07-14 16:14:15.503735 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -377> 2015-07-14 16:14:15.503770 7f3ea9eea700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e9a0).register_pipe -376> 2015-07-14 16:14:15.503792 7f3ea9eea700 10 -- 172.16.0.50:6800/25422 get_connection osd.8 172.16.5.222:6804/13376 new 0x4ad4000 -375> 2015-07-14 16:14:15.503819 7f3ea9eea700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.222:6804/13376 -- osd_op(mds.0.80:7 200.00000002 [stat] 1.eb272dbb ack+read+rwordered+known_if_redirected e2383) v5 -- ?+0 0x4a5bb80 con 0x4a3e9a0 -374> 2015-07-14 16:14:15.503824 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e9a0).writer: state = connecting policy.server=0 -373> 2015-07-14 16:14:15.503884 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e9a0).connect 0 -372> 2015-07-14 16:14:15.503913 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3e9a0).connecting to 172.16.5.222:6804/13376 -371> 2015-07-14 16:14:15.504543 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader wants 195 from dispatch throttler 0/104857600 -370> 2015-07-14 16:14:15.504590 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).aborted = 0 -369> 2015-07-14 16:14:15.504567 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=1 pgs=0 cs=0 l=1 c=0x4a3e9a0).connect sent my addr 172.16.0.50:6800/25422 -368> 2015-07-14 16:14:15.504620 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=1 pgs=0 cs=0 l=1 c=0x4a3e9a0).connect.authorizer_len=25 protocol=1 -367> 2015-07-14 16:14:15.504620 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader got message 4 0x4a762c0 osd_op_reply(6 200.00000001 [stat] v0'0 uv9057 ondisk = 0) v6 -366> 2015-07-14 16:14:15.504632 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=1 pgs=0 cs=0 l=1 c=0x4a3e9a0).connect sending gseq=5 cseq=0 proto=24 -365> 2015-07-14 16:14:15.504636 7f3ea61de700 1 -- 172.16.0.50:6800/25422 <== osd.5 172.16.5.230:6808/27037 4 ==== osd_op_reply(6 200.00000001 [stat] v0'0 uv9057 ondisk = 0) v6 ==== 179+0+16 (1905835529 0 1649499127) 0x4a762c0 con 0x4a3e420 -364> 2015-07-14 16:14:15.504673 7f3ea61de700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 195 to dispatch throttler 195/104857600 -363> 2015-07-14 16:14:15.504663 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -362> 2015-07-14 16:14:15.504707 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).write_ack 4 -361> 2015-07-14 16:14:15.504727 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -360> 2015-07-14 16:14:15.504996 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).connect success 1, lossy = 1, features 1125899906842623 -359> 2015-07-14 16:14:15.505079 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -358> 2015-07-14 16:14:15.505139 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -357> 2015-07-14 16:14:15.508798 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).reader wants 179 from dispatch throttler 0/104857600 -356> 2015-07-14 16:14:15.508852 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).aborted = 0 -355> 2015-07-14 16:14:15.508882 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).reader got message 1 0x4a76840 osd_op_reply(7 200.00000002 [stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 -354> 2015-07-14 16:14:15.508921 7f3ea5edb700 1 -- 172.16.0.50:6800/25422 <== osd.8 172.16.5.222:6804/13376 1 ==== osd_op_reply(7 200.00000002 [stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 179+0+0 (1369052365 0 0) 0x4a76840 con 0x4a3e9a0 -353> 2015-07-14 16:14:15.508926 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -352> 2015-07-14 16:14:15.508960 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 179 to dispatch throttler 179/104857600 -351> 2015-07-14 16:14:15.508957 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).write_ack 1 -350> 2015-07-14 16:14:15.508990 7f3ea9eea700 1 mds.0.journaler(ro) _finish_probe_end write_pos = 4223374 (header had 4221582). recovered. -349> 2015-07-14 16:14:15.508989 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -348> 2015-07-14 16:14:15.509069 7f3ea73e2700 4 mds.0.log Journal 200 recovered. -347> 2015-07-14 16:14:15.509093 7f3ea73e2700 4 mds.0.log Recovered journal 200 in format 1 -346> 2015-07-14 16:14:15.509104 7f3ea73e2700 2 mds.0.80 boot_start 1: loading/discovering base inodes -345> 2015-07-14 16:14:15.509126 7f3ea73e2700 0 mds.0.cache creating system inode with ino:100 -344> 2015-07-14 16:14:15.509286 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.230:6808/27037 -- osd_op(mds.0.80:8 100.00000000 [getxattr inode] 1.c5265ab3 ack+read+known_if_redirected e2383) v5 -- ?+0 0x4a5af00 con 0x4a3e420 -343> 2015-07-14 16:14:15.509368 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6808/9971 -- osd_op(mds.0.80:9 100.00000000.inode [read 0~0] 1.85dde07f ack+read+known_if_redirected e2383) v5 -- ?+0 0x4afc500 con 0x4a3e580 -342> 2015-07-14 16:14:15.509404 7f3ea73e2700 0 mds.0.cache creating system inode with ino:1 -341> 2015-07-14 16:14:15.509391 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -340> 2015-07-14 16:14:15.509435 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -339> 2015-07-14 16:14:15.509446 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6808/9971 -- osd_op(mds.0.80:10 1.00000000 [getxattr inode] 1.6b2cdaff ack+read+known_if_redirected e2383) v5 -- ?+0 0x4afc280 con 0x4a3e580 -338> 2015-07-14 16:14:15.509473 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -337> 2015-07-14 16:14:15.509533 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -336> 2015-07-14 16:14:15.509543 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.230:6808/27037 -- osd_op(mds.0.80:11 1.00000000.inode [read 0~0] 1.232c0e14 ack+read+known_if_redirected e2383) v5 -- ?+0 0x4afc000 con 0x4a3e420 -335> 2015-07-14 16:14:15.509583 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -334> 2015-07-14 16:14:15.509598 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -333> 2015-07-14 16:14:15.509665 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -332> 2015-07-14 16:14:15.510441 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).reader wants 667 from dispatch throttler 0/104857600 -331> 2015-07-14 16:14:15.510449 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader wants 179 from dispatch throttler 0/104857600 -330> 2015-07-14 16:14:15.510483 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).aborted = 0 -329> 2015-07-14 16:14:15.510485 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).aborted = 0 -328> 2015-07-14 16:14:15.510532 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader got message 5 0x4a762c0 osd_op_reply(8 100.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v6 -327> 2015-07-14 16:14:15.510558 7f3ea61de700 1 -- 172.16.0.50:6800/25422 <== osd.5 172.16.5.230:6808/27037 5 ==== osd_op_reply(8 100.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v6 ==== 179+0+0 (2130378456 0 0) 0x4a762c0 con 0x4a3e420 -326> 2015-07-14 16:14:15.510535 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).reader got message 2 0x4a76b00 osd_op_reply(9 100.00000000.inode [read 0~482] v0'0 uv3402 ondisk = 0) v6 -325> 2015-07-14 16:14:15.510579 7f3ea60dd700 1 -- 172.16.0.50:6800/25422 <== osd.7 172.16.5.170:6808/9971 2 ==== osd_op_reply(9 100.00000000.inode [read 0~482] v0'0 uv3402 ondisk = 0) v6 ==== 185+0+482 (2098096564 0 1083596494) 0x4a76b00 con 0x4a3e580 -324> 2015-07-14 16:14:15.510577 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -323> 2015-07-14 16:14:15.510607 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).write_ack 5 -322> 2015-07-14 16:14:15.510588 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -321> 2015-07-14 16:14:15.510610 7f3ea61de700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 179 to dispatch throttler 846/104857600 -320> 2015-07-14 16:14:15.510621 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).write_ack 2 -319> 2015-07-14 16:14:15.510627 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -318> 2015-07-14 16:14:15.510629 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 667 to dispatch throttler 667/104857600 -317> 2015-07-14 16:14:15.510622 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader wants 665 from dispatch throttler 667/104857600 -316> 2015-07-14 16:14:15.510642 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -315> 2015-07-14 16:14:15.510651 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).reader wants 177 from dispatch throttler 665/104857600 -314> 2015-07-14 16:14:15.510648 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).aborted = 0 -313> 2015-07-14 16:14:15.510667 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).aborted = 0 -312> 2015-07-14 16:14:15.510684 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader got message 6 0x4a762c0 osd_op_reply(11 1.00000000.inode [read 0~482] v0'0 uv9031 ondisk = 0) v6 -311> 2015-07-14 16:14:15.510688 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).reader got message 3 0x4a76b00 osd_op_reply(10 1.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v6 -310> 2015-07-14 16:14:15.510715 7f3ea60dd700 1 -- 172.16.0.50:6800/25422 <== osd.7 172.16.5.170:6808/9971 3 ==== osd_op_reply(10 1.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v6 ==== 177+0+0 (3581230196 0 0) 0x4a76b00 con 0x4a3e580 -309> 2015-07-14 16:14:15.510713 7f3ea61de700 1 -- 172.16.0.50:6800/25422 <== osd.5 172.16.5.230:6808/27037 6 ==== osd_op_reply(11 1.00000000.inode [read 0~482] v0'0 uv9031 ondisk = 0) v6 ==== 183+0+482 (3734016473 0 1931371576) 0x4a762c0 con 0x4a3e420 -308> 2015-07-14 16:14:15.510722 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -307> 2015-07-14 16:14:15.510725 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -306> 2015-07-14 16:14:15.510739 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).write_ack 6 -305> 2015-07-14 16:14:15.510740 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).write_ack 3 -304> 2015-07-14 16:14:15.510746 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 177 to dispatch throttler 842/104857600 -303> 2015-07-14 16:14:15.510751 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -302> 2015-07-14 16:14:15.510753 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -301> 2015-07-14 16:14:15.510759 7f3ea61de700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 665 to dispatch throttler 665/104857600 -300> 2015-07-14 16:14:15.510786 7f3ea9eea700 2 mds.0.80 boot_start 2: replaying mds log -299> 2015-07-14 16:14:15.510963 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.230:6808/27037 -- osd_op(mds.0.80:12 200.00000001 [read 0~29070] 1.6e5f474 ack+read+known_if_redirected e2383) v5 -- ?+0 0x4a5b180 con 0x4a3e420 -298> 2015-07-14 16:14:15.511052 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -297> 2015-07-14 16:14:15.511131 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -296> 2015-07-14 16:14:15.511987 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader wants 29249 from dispatch throttler 0/104857600 -295> 2015-07-14 16:14:15.512478 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).aborted = 0 -294> 2015-07-14 16:14:15.512544 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader got message 7 0x4a762c0 osd_op_reply(12 200.00000001 [read 0~29070] v0'0 uv9057 ondisk = 0) v6 -293> 2015-07-14 16:14:15.512565 7f3ea61de700 1 -- 172.16.0.50:6800/25422 <== osd.5 172.16.5.230:6808/27037 7 ==== osd_op_reply(12 200.00000001 [read 0~29070] v0'0 uv9057 ondisk = 0) v6 ==== 179+0+29070 (3485488834 0 3168097469) 0x4a762c0 con 0x4a3e420 -292> 2015-07-14 16:14:15.512583 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -291> 2015-07-14 16:14:15.512602 7f3ea61de700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 29249 to dispatch throttler 29249/104857600 -290> 2015-07-14 16:14:15.512614 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).write_ack 7 -289> 2015-07-14 16:14:15.512635 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -288> 2015-07-14 16:14:15.514747 7f3ea73e2700 1 mds.0.80 replay_done -287> 2015-07-14 16:14:15.514763 7f3ea73e2700 1 mds.0.80 making mds journal writeable -286> 2015-07-14 16:14:15.514767 7f3ea73e2700 1 mds.0.journaler(ro) set_writeable -285> 2015-07-14 16:14:15.514824 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.230:6808/27037 -- osd_op(mds.0.80:13 200.00000001 [zero 29070~4165234] 1.6e5f474 ondisk+write+known_if_redirected e2383) v5 -- ?+0 0x4a5b680 con 0x4a3e420 -284> 2015-07-14 16:14:15.514883 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.222:6804/13376 -- osd_op(mds.0.80:14 200.00000002 [delete] 1.eb272dbb ondisk+write+known_if_redirected e2383) v5 -- ?+0 0x4afc780 con 0x4a3e9a0 -283> 2015-07-14 16:14:15.514902 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -282> 2015-07-14 16:14:15.514931 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.0.50:6801/15304 -- osd_op(mds.0.80:15 200.00000003 [delete] 1.95e3ab ondisk+write+known_if_redirected e2383) v5 -- ?+0 0x4afc500 con 0x4a3e6e0 -281> 2015-07-14 16:14:15.514959 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -280> 2015-07-14 16:14:15.514972 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6808/9971 -- osd_op(mds.0.80:16 200.00000004 [delete] 1.1ba6bc1f ondisk+write+known_if_redirected e2383) v5 -- ?+0 0x4a5af00 con 0x4a3e580 -279> 2015-07-14 16:14:15.514978 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -278> 2015-07-14 16:14:15.514967 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).writer: state = open policy.server=0 -277> 2015-07-14 16:14:15.515029 7f3ea73e2700 10 -- 172.16.0.50:6800/25422 connect_rank to 172.16.5.222:6800/3627, creating pipe and registering -276> 2015-07-14 16:14:15.515019 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -275> 2015-07-14 16:14:15.515049 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -274> 2015-07-14 16:14:15.515087 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).writer: state = open policy.server=0 -273> 2015-07-14 16:14:15.515094 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -272> 2015-07-14 16:14:15.515112 7f3ea73e2700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3eb00).register_pipe -271> 2015-07-14 16:14:15.515136 7f3ea73e2700 10 -- 172.16.0.50:6800/25422 get_connection osd.4 172.16.5.222:6800/3627 new 0x4ad9000 -270> 2015-07-14 16:14:15.515157 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.222:6800/3627 -- osd_op(mds.0.80:17 200.00000005 [delete] 1.96d5827a ondisk+write+known_if_redirected e2383) v5 -- ?+0 0x4afc780 con 0x4a3eb00 -269> 2015-07-14 16:14:15.515151 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3eb00).writer: state = connecting policy.server=0 -268> 2015-07-14 16:14:15.515200 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3eb00).connect 0 -267> 2015-07-14 16:14:15.515236 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :0 s=1 pgs=0 cs=0 l=1 c=0x4a3eb00).connecting to 172.16.5.222:6800/3627 -266> 2015-07-14 16:14:15.515273 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.222:6804/13376 -- osd_op(mds.0.80:18 200.00000006 [delete] 1.ade7efde ondisk+write+known_if_redirected e2383) v5 -- ?+0 0x4afc500 con 0x4a3e9a0 -265> 2015-07-14 16:14:15.515315 7f3ea73e2700 2 mds.0.80 i am alone, moving to state reconnect -264> 2015-07-14 16:14:15.515321 7f3ea73e2700 3 mds.0.80 request_state up:reconnect -263> 2015-07-14 16:14:15.515325 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -262> 2015-07-14 16:14:15.515344 7f3ea73e2700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -261> 2015-07-14 16:14:15.515353 7f3ea73e2700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 2 v512) v3 -- ?+0 0x4a96900 con 0x4a3e160 -260> 2015-07-14 16:14:15.515380 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -259> 2015-07-14 16:14:15.515414 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -258> 2015-07-14 16:14:15.515498 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -257> 2015-07-14 16:14:15.515677 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader wants 179 from dispatch throttler 0/104857600 -256> 2015-07-14 16:14:15.515713 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).aborted = 0 -255> 2015-07-14 16:14:15.515768 7f3ea62df700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).reader wants 179 from dispatch throttler 179/104857600 -254> 2015-07-14 16:14:15.515813 7f3ea62df700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).aborted = 0 -253> 2015-07-14 16:14:15.515773 7f3ea61de700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).reader got message 8 0x4a762c0 osd_op_reply(13 200.00000001 [zero 29070~4165234] v0'0 uv9057 ondisk = 0) v6 -252> 2015-07-14 16:14:15.515832 7f3ea61de700 1 -- 172.16.0.50:6800/25422 <== osd.5 172.16.5.230:6808/27037 8 ==== osd_op_reply(13 200.00000001 [zero 29070~4165234] v0'0 uv9057 ondisk = 0) v6 ==== 179+0+0 (4120333419 0 0) 0x4a762c0 con 0x4a3e420 -251> 2015-07-14 16:14:15.515843 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -250> 2015-07-14 16:14:15.515847 7f3ea62df700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).reader got message 2 0x4a76000 osd_op_reply(15 200.00000003 [delete] v0'0 uv3088 ondisk = -2 ((2) No such file or directory)) v6 -249> 2015-07-14 16:14:15.515874 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).write_ack 8 -248> 2015-07-14 16:14:15.515857 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).reader wants 179 from dispatch throttler 358/104857600 -247> 2015-07-14 16:14:15.515880 7f3ea61de700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 179 to dispatch throttler 358/104857600 -246> 2015-07-14 16:14:15.515852 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=1 pgs=0 cs=0 l=1 c=0x4a3eb00).connect sent my addr 172.16.0.50:6800/25422 -245> 2015-07-14 16:14:15.515880 7f3ea62df700 1 -- 172.16.0.50:6800/25422 <== osd.0 172.16.0.50:6801/15304 2 ==== osd_op_reply(15 200.00000003 [delete] v0'0 uv3088 ondisk = -2 ((2) No such file or directory)) v6 ==== 179+0+0 (3128093706 0 0) 0x4a76000 con 0x4a3e6e0 -244> 2015-07-14 16:14:15.515893 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).aborted = 0 -243> 2015-07-14 16:14:15.515896 7f3ea75e4700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.230:6808/27037 pipe(0x4a83000 sd=17 :56248 s=2 pgs=11768 cs=1 l=1 c=0x4a3e420).writer: state = open policy.server=0 -242> 2015-07-14 16:14:15.515894 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).writer: state = open policy.server=0 -241> 2015-07-14 16:14:15.515926 7f3ea62df700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 179 to dispatch throttler 358/104857600 -240> 2015-07-14 16:14:15.515927 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).write_ack 2 -239> 2015-07-14 16:14:15.515929 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=1 pgs=0 cs=0 l=1 c=0x4a3eb00).connect.authorizer_len=25 protocol=1 -238> 2015-07-14 16:14:15.515953 7f3ea74e3700 10 -- 172.16.0.50:6800/25422 >> 172.16.0.50:6801/15304 pipe(0x4ab5000 sd=18 :53011 s=2 pgs=17 cs=1 l=1 c=0x4a3e6e0).writer: state = open policy.server=0 -237> 2015-07-14 16:14:15.515924 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).reader got message 4 0x4a76b00 osd_op_reply(16 200.00000004 [delete] v0'0 uv3402 ondisk = -2 ((2) No such file or directory)) v6 -236> 2015-07-14 16:14:15.515951 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=1 pgs=0 cs=0 l=1 c=0x4a3eb00).connect sending gseq=6 cseq=0 proto=24 -235> 2015-07-14 16:14:15.515971 7f3ea60dd700 1 -- 172.16.0.50:6800/25422 <== osd.7 172.16.5.170:6808/9971 4 ==== osd_op_reply(16 200.00000004 [delete] v0'0 uv3402 ondisk = -2 ((2) No such file or directory)) v6 ==== 179+0+0 (3547337967 0 0) 0x4a76b00 con 0x4a3e580 -234> 2015-07-14 16:14:15.515981 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -233> 2015-07-14 16:14:15.515996 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).write_ack 4 -232> 2015-07-14 16:14:15.516011 7f3ea60dd700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 179 to dispatch throttler 179/104857600 -231> 2015-07-14 16:14:15.516010 7f3ea63e0700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6808/9971 pipe(0x4ab0000 sd=19 :44705 s=2 pgs=19 cs=1 l=1 c=0x4a3e580).writer: state = open policy.server=0 -230> 2015-07-14 16:14:15.516037 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).reader wants 179 from dispatch throttler 0/104857600 -229> 2015-07-14 16:14:15.516067 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).aborted = 0 -228> 2015-07-14 16:14:15.516093 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).reader got message 2 0x4a76840 osd_op_reply(14 200.00000002 [delete] v0'0 uv11656 ondisk = -2 ((2) No such file or directory)) v6 -227> 2015-07-14 16:14:15.516148 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -226> 2015-07-14 16:14:15.516139 7f3ea5edb700 1 -- 172.16.0.50:6800/25422 <== osd.8 172.16.5.222:6804/13376 2 ==== osd_op_reply(14 200.00000002 [delete] v0'0 uv11656 ondisk = -2 ((2) No such file or directory)) v6 ==== 179+0+0 (690670509 0 0) 0x4a76840 con 0x4a3e9a0 -225> 2015-07-14 16:14:15.516167 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).write_ack 2 -224> 2015-07-14 16:14:15.516188 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 179 to dispatch throttler 179/104857600 -223> 2015-07-14 16:14:15.516183 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -222> 2015-07-14 16:14:15.516268 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=2 pgs=81 cs=1 l=1 c=0x4a3eb00).connect success 1, lossy = 1, features 1125899906842623 -221> 2015-07-14 16:14:15.516325 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).reader wants 179 from dispatch throttler 0/104857600 -220> 2015-07-14 16:14:15.516354 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).aborted = 0 -219> 2015-07-14 16:14:15.516386 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=2 pgs=81 cs=1 l=1 c=0x4a3eb00).writer: state = open policy.server=0 -218> 2015-07-14 16:14:15.516386 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).reader got message 3 0x4a76840 osd_op_reply(18 200.00000006 [delete] v0'0 uv8643 ondisk = -2 ((2) No such file or directory)) v6 -217> 2015-07-14 16:14:15.516413 7f3ea5edb700 1 -- 172.16.0.50:6800/25422 <== osd.8 172.16.5.222:6804/13376 3 ==== osd_op_reply(18 200.00000006 [delete] v0'0 uv8643 ondisk = -2 ((2) No such file or directory)) v6 ==== 179+0+0 (1264564127 0 0) 0x4a76840 con 0x4a3e9a0 -216> 2015-07-14 16:14:15.516428 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -215> 2015-07-14 16:14:15.516454 7f3ea5edb700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 179 to dispatch throttler 179/104857600 -214> 2015-07-14 16:14:15.516458 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).write_ack 3 -213> 2015-07-14 16:14:15.516457 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=2 pgs=81 cs=1 l=1 c=0x4a3eb00).writer: state = open policy.server=0 -212> 2015-07-14 16:14:15.516477 7f3ea5fdc700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6804/13376 pipe(0x4ad4000 sd=20 :44210 s=2 pgs=69 cs=1 l=1 c=0x4a3e9a0).writer: state = open policy.server=0 -211> 2015-07-14 16:14:15.519205 7f3ea5cd9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=2 pgs=81 cs=1 l=1 c=0x4a3eb00).reader wants 179 from dispatch throttler 0/104857600 -210> 2015-07-14 16:14:15.519251 7f3ea5cd9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=2 pgs=81 cs=1 l=1 c=0x4a3eb00).aborted = 0 -209> 2015-07-14 16:14:15.519277 7f3ea5cd9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=2 pgs=81 cs=1 l=1 c=0x4a3eb00).reader got message 1 0x4a77340 osd_op_reply(17 200.00000005 [delete] v0'0 uv7448 ondisk = -2 ((2) No such file or directory)) v6 -208> 2015-07-14 16:14:15.519303 7f3ea5cd9700 1 -- 172.16.0.50:6800/25422 <== osd.4 172.16.5.222:6800/3627 1 ==== osd_op_reply(17 200.00000005 [delete] v0'0 uv7448 ondisk = -2 ((2) No such file or directory)) v6 ==== 179+0+0 (74902788 0 0) 0x4a77340 con 0x4a3eb00 -207> 2015-07-14 16:14:15.519336 7f3ea5cd9700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 179 to dispatch throttler 179/104857600 -206> 2015-07-14 16:14:15.519321 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=2 pgs=81 cs=1 l=1 c=0x4a3eb00).writer: state = open policy.server=0 -205> 2015-07-14 16:14:15.519360 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=2 pgs=81 cs=1 l=1 c=0x4a3eb00).write_ack 1 -204> 2015-07-14 16:14:15.519390 7f3ea5dda700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.222:6800/3627 pipe(0x4ad9000 sd=21 :42408 s=2 pgs=81 cs=1 l=1 c=0x4a3eb00).writer: state = open policy.server=0 -203> 2015-07-14 16:14:16.572942 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 680 from dispatch throttler 0/104857600 -202> 2015-07-14 16:14:16.572985 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -201> 2015-07-14 16:14:16.573003 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -200> 2015-07-14 16:14:16.573019 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 17 0x4aeb200 mdsmap(e 513) v1 -199> 2015-07-14 16:14:16.573065 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -198> 2015-07-14 16:14:16.573095 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 17 -197> 2015-07-14 16:14:16.573117 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -196> 2015-07-14 16:14:16.573109 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 17 ==== mdsmap(e 513) v1 ==== 680+0+0 (555836137 0 0) 0x4aeb200 con 0x4a3e160 -195> 2015-07-14 16:14:16.573144 7f3eac6ef700 5 mds.0.80 handle_mds_map epoch 513 from mon.2 -194> 2015-07-14 16:14:16.573197 7f3eac6ef700 1 mds.0.80 handle_mds_map i am now mds.0.80 -193> 2015-07-14 16:14:16.573230 7f3eac6ef700 1 mds.0.80 handle_mds_map state change up:replay --> up:reconnect -192> 2015-07-14 16:14:16.573234 7f3eac6ef700 1 mds.0.80 reconnect_start -191> 2015-07-14 16:14:16.573237 7f3eac6ef700 1 mds.0.80 reopen_log -190> 2015-07-14 16:14:16.573252 7f3eac6ef700 1 mds.0.server reconnect_clients -- 6 sessions -189> 2015-07-14 16:14:16.573312 7f3eac6ef700 5 mds.0.bal rebalance done -188> 2015-07-14 16:14:16.573341 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 680 to dispatch throttler 680/104857600 -187> 2015-07-14 16:14:18.031180 7f3eaaeec700 10 monclient: tick -186> 2015-07-14 16:14:18.031206 7f3eaaeec700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2015-07-14 16:13:48.031203) -185> 2015-07-14 16:14:18.031240 7f3eaaeec700 10 monclient: renew subs? (now: 2015-07-14 16:14:18.031239; renew after: 2015-07-14 16:16:45.170498) -- no -184> 2015-07-14 16:14:18.031306 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -183> 2015-07-14 16:14:18.031346 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_keepalive2 14 2015-07-14 16:14:18.031345 -182> 2015-07-14 16:14:18.031394 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -181> 2015-07-14 16:14:19.515467 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -180> 2015-07-14 16:14:19.515498 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 3 v513) v3 -- ?+0 0x4a96000 con 0x4a3e160 -179> 2015-07-14 16:14:19.515592 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -178> 2015-07-14 16:14:19.515722 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -177> 2015-07-14 16:14:19.517596 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -176> 2015-07-14 16:14:19.517629 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -175> 2015-07-14 16:14:19.517648 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -174> 2015-07-14 16:14:19.517659 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 18 0x4a96900 mdsbeacon(94247/codetest up:reconnect seq 3 v513) v3 -173> 2015-07-14 16:14:19.517695 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -172> 2015-07-14 16:14:19.517716 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 18 -171> 2015-07-14 16:14:19.517734 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -170> 2015-07-14 16:14:19.517744 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 18 ==== mdsbeacon(94247/codetest up:reconnect seq 3 v513) v3 ==== 120+0+0 (482147640 0 0) 0x4a96900 con 0x4a3e160 -169> 2015-07-14 16:14:19.517804 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -168> 2015-07-14 16:14:23.515558 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -167> 2015-07-14 16:14:23.515583 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 4 v513) v3 -- ?+0 0x4a96f00 con 0x4a3e160 -166> 2015-07-14 16:14:23.515657 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -165> 2015-07-14 16:14:23.515742 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -164> 2015-07-14 16:14:23.517375 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -163> 2015-07-14 16:14:23.517417 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -162> 2015-07-14 16:14:23.517439 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -161> 2015-07-14 16:14:23.517477 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 19 0x4a97500 mdsbeacon(94247/codetest up:reconnect seq 4 v513) v3 -160> 2015-07-14 16:14:23.517517 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -159> 2015-07-14 16:14:23.517548 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 19 -158> 2015-07-14 16:14:23.517568 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -157> 2015-07-14 16:14:23.517564 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 19 ==== mdsbeacon(94247/codetest up:reconnect seq 4 v513) v3 ==== 120+0+0 (4023399599 0 0) 0x4a97500 con 0x4a3e160 -156> 2015-07-14 16:14:23.517601 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -155> 2015-07-14 16:14:27.515637 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -154> 2015-07-14 16:14:27.515663 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 5 v513) v3 -- ?+0 0x4a96c00 con 0x4a3e160 -153> 2015-07-14 16:14:27.515698 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -152> 2015-07-14 16:14:27.515782 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -151> 2015-07-14 16:14:27.517403 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -150> 2015-07-14 16:14:27.517446 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -149> 2015-07-14 16:14:27.517468 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -148> 2015-07-14 16:14:27.517482 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 20 0x4a97200 mdsbeacon(94247/codetest up:reconnect seq 5 v513) v3 -147> 2015-07-14 16:14:27.517524 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -146> 2015-07-14 16:14:27.517554 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 20 -145> 2015-07-14 16:14:27.517577 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -144> 2015-07-14 16:14:27.517566 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 20 ==== mdsbeacon(94247/codetest up:reconnect seq 5 v513) v3 ==== 120+0+0 (288838920 0 0) 0x4a97200 con 0x4a3e160 -143> 2015-07-14 16:14:27.517603 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -142> 2015-07-14 16:14:28.031328 7f3eaaeec700 10 monclient: tick -141> 2015-07-14 16:14:28.031350 7f3eaaeec700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2015-07-14 16:13:58.031348) -140> 2015-07-14 16:14:28.031372 7f3eaaeec700 10 monclient: renew subs? (now: 2015-07-14 16:14:28.031371; renew after: 2015-07-14 16:16:45.170498) -- no -139> 2015-07-14 16:14:28.031432 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -138> 2015-07-14 16:14:28.031470 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_keepalive2 14 2015-07-14 16:14:28.031468 -137> 2015-07-14 16:14:28.031563 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -136> 2015-07-14 16:14:31.515722 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -135> 2015-07-14 16:14:31.515749 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 6 v513) v3 -- ?+0 0x4a96c00 con 0x4a3e160 -134> 2015-07-14 16:14:31.515786 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -133> 2015-07-14 16:14:31.515863 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -132> 2015-07-14 16:14:31.517419 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -131> 2015-07-14 16:14:31.517460 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -130> 2015-07-14 16:14:31.517482 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -129> 2015-07-14 16:14:31.517497 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 21 0x4a97b00 mdsbeacon(94247/codetest up:reconnect seq 6 v513) v3 -128> 2015-07-14 16:14:31.517534 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -127> 2015-07-14 16:14:31.517549 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 21 -126> 2015-07-14 16:14:31.517564 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -125> 2015-07-14 16:14:31.517582 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 21 ==== mdsbeacon(94247/codetest up:reconnect seq 6 v513) v3 ==== 120+0+0 (401768720 0 0) 0x4a97b00 con 0x4a3e160 -124> 2015-07-14 16:14:31.517620 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -123> 2015-07-14 16:14:35.515803 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -122> 2015-07-14 16:14:35.515829 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 7 v513) v3 -- ?+0 0x4a96f00 con 0x4a3e160 -121> 2015-07-14 16:14:35.515894 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -120> 2015-07-14 16:14:35.515997 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -119> 2015-07-14 16:14:35.517577 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -118> 2015-07-14 16:14:35.517623 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -117> 2015-07-14 16:14:35.517645 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -116> 2015-07-14 16:14:35.517659 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 22 0x4a97800 mdsbeacon(94247/codetest up:reconnect seq 7 v513) v3 -115> 2015-07-14 16:14:35.517698 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -114> 2015-07-14 16:14:35.517729 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 22 -113> 2015-07-14 16:14:35.517750 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -112> 2015-07-14 16:14:35.517736 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 22 ==== mdsbeacon(94247/codetest up:reconnect seq 7 v513) v3 ==== 120+0+0 (3910525111 0 0) 0x4a97800 con 0x4a3e160 -111> 2015-07-14 16:14:35.517776 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -110> 2015-07-14 16:14:38.031471 7f3eaaeec700 10 monclient: tick -109> 2015-07-14 16:14:38.031504 7f3eaaeec700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2015-07-14 16:14:08.031502) -108> 2015-07-14 16:14:38.031565 7f3eaaeec700 10 monclient: renew subs? (now: 2015-07-14 16:14:38.031564; renew after: 2015-07-14 16:16:45.170498) -- no -107> 2015-07-14 16:14:38.031639 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -106> 2015-07-14 16:14:38.031696 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_keepalive2 14 2015-07-14 16:14:38.031695 -105> 2015-07-14 16:14:38.031751 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -104> 2015-07-14 16:14:39.515890 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -103> 2015-07-14 16:14:39.515916 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 8 v513) v3 -- ?+0 0x4a96000 con 0x4a3e160 -102> 2015-07-14 16:14:39.515994 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -101> 2015-07-14 16:14:39.516095 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -100> 2015-07-14 16:14:39.517857 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -99> 2015-07-14 16:14:39.517904 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -98> 2015-07-14 16:14:39.517952 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -97> 2015-07-14 16:14:39.517968 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 23 0x4b3c900 mdsbeacon(94247/codetest up:reconnect seq 8 v513) v3 -96> 2015-07-14 16:14:39.518009 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -95> 2015-07-14 16:14:39.518040 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 23 -94> 2015-07-14 16:14:39.518061 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -93> 2015-07-14 16:14:39.518058 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 23 ==== mdsbeacon(94247/codetest up:reconnect seq 8 v513) v3 ==== 120+0+0 (4106708175 0 0) 0x4b3c900 con 0x4a3e160 -92> 2015-07-14 16:14:39.518094 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -91> 2015-07-14 16:14:43.515978 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -90> 2015-07-14 16:14:43.516003 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 9 v513) v3 -- ?+0 0x4b3d500 con 0x4a3e160 -89> 2015-07-14 16:14:43.516078 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -88> 2015-07-14 16:14:43.516176 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -87> 2015-07-14 16:14:43.517877 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -86> 2015-07-14 16:14:43.517921 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -85> 2015-07-14 16:14:43.517942 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -84> 2015-07-14 16:14:43.517956 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 24 0x4b3c600 mdsbeacon(94247/codetest up:reconnect seq 9 v513) v3 -83> 2015-07-14 16:14:43.517996 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -82> 2015-07-14 16:14:43.518027 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 24 -81> 2015-07-14 16:14:43.518047 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -80> 2015-07-14 16:14:43.518043 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 24 ==== mdsbeacon(94247/codetest up:reconnect seq 9 v513) v3 ==== 120+0+0 (169870696 0 0) 0x4b3c600 con 0x4a3e160 -79> 2015-07-14 16:14:43.518096 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -78> 2015-07-14 16:14:47.516054 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -77> 2015-07-14 16:14:47.516088 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 10 v513) v3 -- ?+0 0x4b3d200 con 0x4a3e160 -76> 2015-07-14 16:14:47.516161 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -75> 2015-07-14 16:14:47.516270 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -74> 2015-07-14 16:14:47.517983 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -73> 2015-07-14 16:14:47.518029 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -72> 2015-07-14 16:14:47.518054 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -71> 2015-07-14 16:14:47.518068 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 25 0x4b3c300 mdsbeacon(94247/codetest up:reconnect seq 10 v513) v3 -70> 2015-07-14 16:14:47.518133 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -69> 2015-07-14 16:14:47.518163 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 25 -68> 2015-07-14 16:14:47.518153 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 25 ==== mdsbeacon(94247/codetest up:reconnect seq 10 v513) v3 ==== 120+0+0 (216387952 0 0) 0x4b3c300 con 0x4a3e160 -67> 2015-07-14 16:14:47.518185 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -66> 2015-07-14 16:14:47.518186 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -65> 2015-07-14 16:14:48.031669 7f3eaaeec700 10 monclient: tick -64> 2015-07-14 16:14:48.031696 7f3eaaeec700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2015-07-14 16:14:18.031693) -63> 2015-07-14 16:14:48.031749 7f3eaaeec700 10 monclient: renew subs? (now: 2015-07-14 16:14:48.031749; renew after: 2015-07-14 16:16:45.170498) -- no -62> 2015-07-14 16:14:48.031821 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -61> 2015-07-14 16:14:48.031859 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_keepalive2 14 2015-07-14 16:14:48.031858 -60> 2015-07-14 16:14:48.031909 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -59> 2015-07-14 16:14:51.516177 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -58> 2015-07-14 16:14:51.516208 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 11 v513) v3 -- ?+0 0x4b3cf00 con 0x4a3e160 -57> 2015-07-14 16:14:51.516285 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -56> 2015-07-14 16:14:51.516385 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -55> 2015-07-14 16:14:51.518222 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -54> 2015-07-14 16:14:51.518263 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -53> 2015-07-14 16:14:51.518287 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -52> 2015-07-14 16:14:51.518302 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 26 0x4b3c000 mdsbeacon(94247/codetest up:reconnect seq 11 v513) v3 -51> 2015-07-14 16:14:51.518346 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -50> 2015-07-14 16:14:51.518377 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 26 -49> 2015-07-14 16:14:51.518398 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -48> 2015-07-14 16:14:51.518400 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 26 ==== mdsbeacon(94247/codetest up:reconnect seq 11 v513) v3 ==== 120+0+0 (4060262615 0 0) 0x4b3c000 con 0x4a3e160 -47> 2015-07-14 16:14:51.518451 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -46> 2015-07-14 16:14:55.516243 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -45> 2015-07-14 16:14:55.516268 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 12 v513) v3 -- ?+0 0x4b3cc00 con 0x4a3e160 -44> 2015-07-14 16:14:55.516344 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -43> 2015-07-14 16:14:55.516426 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -42> 2015-07-14 16:14:55.518102 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -41> 2015-07-14 16:14:55.518170 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -40> 2015-07-14 16:14:55.518194 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -39> 2015-07-14 16:14:55.518209 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 27 0x4b3d800 mdsbeacon(94247/codetest up:reconnect seq 12 v513) v3 -38> 2015-07-14 16:14:55.518249 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -37> 2015-07-14 16:14:55.518280 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 27 -36> 2015-07-14 16:14:55.518300 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -35> 2015-07-14 16:14:55.518294 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 27 ==== mdsbeacon(94247/codetest up:reconnect seq 12 v513) v3 ==== 120+0+0 (24017216 0 0) 0x4b3d800 con 0x4a3e160 -34> 2015-07-14 16:14:55.518343 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -33> 2015-07-14 16:14:58.031869 7f3eaaeec700 10 monclient: tick -32> 2015-07-14 16:14:58.031890 7f3eaaeec700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2015-07-14 16:14:28.031887) -31> 2015-07-14 16:14:58.031917 7f3eaaeec700 10 monclient: renew subs? (now: 2015-07-14 16:14:58.031916; renew after: 2015-07-14 16:16:45.170498) -- no -30> 2015-07-14 16:14:58.031980 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -29> 2015-07-14 16:14:58.032017 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_keepalive2 14 2015-07-14 16:14:58.032016 -28> 2015-07-14 16:14:58.032089 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -27> 2015-07-14 16:14:59.516336 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -26> 2015-07-14 16:14:59.516361 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 13 v513) v3 -- ?+0 0x4b3db00 con 0x4a3e160 -25> 2015-07-14 16:14:59.516433 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -24> 2015-07-14 16:14:59.516528 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -23> 2015-07-14 16:14:59.518354 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -22> 2015-07-14 16:14:59.518395 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -21> 2015-07-14 16:14:59.518417 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -20> 2015-07-14 16:14:59.518432 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 28 0x4b3d500 mdsbeacon(94247/codetest up:reconnect seq 13 v513) v3 -19> 2015-07-14 16:14:59.518470 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -18> 2015-07-14 16:14:59.518500 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 28 -17> 2015-07-14 16:14:59.518521 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -16> 2015-07-14 16:14:59.518522 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 28 ==== mdsbeacon(94247/codetest up:reconnect seq 13 v513) v3 ==== 120+0+0 (4287174887 0 0) 0x4b3d500 con 0x4a3e160 -15> 2015-07-14 16:14:59.518570 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -14> 2015-07-14 16:15:03.516438 7f3ea8de7700 10 monclient: _send_mon_message to mon.cloudCode04 at 172.16.5.170:6789/0 -13> 2015-07-14 16:15:03.516471 7f3ea8de7700 1 -- 172.16.0.50:6800/25422 --> 172.16.5.170:6789/0 -- mdsbeacon(94247/codetest up:reconnect seq 14 v513) v3 -- ?+0 0x4b46f00 con 0x4a3e160 -12> 2015-07-14 16:15:03.516539 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -11> 2015-07-14 16:15:03.516642 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -10> 2015-07-14 16:15:03.518299 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader wants 120 from dispatch throttler 0/104857600 -9> 2015-07-14 16:15:03.518348 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).aborted = 0 -8> 2015-07-14 16:15:03.518372 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).No session security set -7> 2015-07-14 16:15:03.518389 7f3ea96e9700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).reader got message 29 0x4a96000 mdsbeacon(94247/codetest up:reconnect seq 14 v513) v3 -6> 2015-07-14 16:15:03.518428 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -5> 2015-07-14 16:15:03.518459 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).write_ack 29 -4> 2015-07-14 16:15:03.518481 7f3eb13d8700 10 -- 172.16.0.50:6800/25422 >> 172.16.5.170:6789/0 pipe(0x4a7e000 sd=8 :36805 s=2 pgs=6011 cs=1 l=1 c=0x4a3e160).writer: state = open policy.server=0 -3> 2015-07-14 16:15:03.518485 7f3eac6ef700 1 -- 172.16.0.50:6800/25422 <== mon.2 172.16.5.170:6789/0 29 ==== mdsbeacon(94247/codetest up:reconnect seq 14 v513) v3 ==== 120+0+0 (4182559999 0 0) 0x4a96000 con 0x4a3e160 -2> 2015-07-14 16:15:03.518537 7f3eac6ef700 10 -- 172.16.0.50:6800/25422 dispatch_throttle_release 120 to dispatch throttler 120/104857600 -1> 2015-07-14 16:15:05.036630 7f3ea95e8700 1 mds.0.server reconnect gave up on client.4254 172.16.0.60:0/17315 0> 2015-07-14 16:15:05.043112 7f3ea95e8700 -1 ./include/interval_set.h: In function 'void interval_set::insert(T, T) [with T = inodeno_t]' thread 7f3ea95e8700 time 2015-07-14 16:15:05.036693 ./include/interval_set.h: 340: FAILED assert(0) ceph version 9.0.0 (bd7989103911796eb5698cf208b0ccdc3370d707) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x72) [0xa623c2] 2: /usr/bin/ceph-mds() [0x69e82a] 3: (interval_set::insert(interval_set const&)+0x1eb) [0x744e7b] 4: (InoTable::project_release_ids(interval_set&)+0x73) [0x8c3893] 5: (Server::journal_close_session(Session*, int, Context*)+0x168) [0x70d0d8] 6: (Server::kill_session(Session*, Context*)+0x13b) [0x70d69b] 7: (Server::reconnect_tick()+0x128) [0x70d9d8] 8: (MDS::tick()+0x3fc) [0x6b1e0c] 9: (MDSInternalContextBase::complete(int)+0x15b) [0x8e6f2b] 10: (SafeTimer::timer_thread()+0x1af) [0xa5496f] 11: (SafeTimerThread::entry()+0xd) [0xa55f2d] 12: (()+0x6b50) [0x7f3eb0d70b50] 13: (clone()+0x6d) [0x7f3eaf8127bd] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 10/10 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mds.codetest.log --- end dump of recent events --- 2015-07-14 16:15:05.065169 7f3ea95e8700 -1 *** Caught signal (Aborted) ** in thread 7f3ea95e8700 ceph version 9.0.0 (bd7989103911796eb5698cf208b0ccdc3370d707) 1: /usr/bin/ceph-mds() [0x97dcac] 2: (()+0xf0a0) [0x7f3eb0d790a0] 3: (gsignal()+0x35) [0x7f3eaf768165] 4: (abort()+0x180) [0x7f3eaf76b3e0] 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f3eaffbf89d] 6: (()+0x63996) [0x7f3eaffbd996] 7: (()+0x639c3) [0x7f3eaffbd9c3] 8: (()+0x63bee) [0x7f3eaffbdbee] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x220) [0xa62570] 10: /usr/bin/ceph-mds() [0x69e82a] 11: (interval_set::insert(interval_set const&)+0x1eb) [0x744e7b] 12: (InoTable::project_release_ids(interval_set&)+0x73) [0x8c3893] 13: (Server::journal_close_session(Session*, int, Context*)+0x168) [0x70d0d8] 14: (Server::kill_session(Session*, Context*)+0x13b) [0x70d69b] 15: (Server::reconnect_tick()+0x128) [0x70d9d8] 16: (MDS::tick()+0x3fc) [0x6b1e0c] 17: (MDSInternalContextBase::complete(int)+0x15b) [0x8e6f2b] 18: (SafeTimer::timer_thread()+0x1af) [0xa5496f] 19: (SafeTimerThread::entry()+0xd) [0xa55f2d] 20: (()+0x6b50) [0x7f3eb0d70b50] 21: (clone()+0x6d) [0x7f3eaf8127bd] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-07-14 16:15:05.065169 7f3ea95e8700 -1 *** Caught signal (Aborted) ** in thread 7f3ea95e8700 ceph version 9.0.0 (bd7989103911796eb5698cf208b0ccdc3370d707) 1: /usr/bin/ceph-mds() [0x97dcac] 2: (()+0xf0a0) [0x7f3eb0d790a0] 3: (gsignal()+0x35) [0x7f3eaf768165] 4: (abort()+0x180) [0x7f3eaf76b3e0] 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f3eaffbf89d] 6: (()+0x63996) [0x7f3eaffbd996] 7: (()+0x639c3) [0x7f3eaffbd9c3] 8: (()+0x63bee) [0x7f3eaffbdbee] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x220) [0xa62570] 10: /usr/bin/ceph-mds() [0x69e82a] 11: (interval_set::insert(interval_set const&)+0x1eb) [0x744e7b] 12: (InoTable::project_release_ids(interval_set&)+0x73) [0x8c3893] 13: (Server::journal_close_session(Session*, int, Context*)+0x168) [0x70d0d8] 14: (Server::kill_session(Session*, Context*)+0x13b) [0x70d69b] 15: (Server::reconnect_tick()+0x128) [0x70d9d8] 16: (MDS::tick()+0x3fc) [0x6b1e0c] 17: (MDSInternalContextBase::complete(int)+0x15b) [0x8e6f2b] 18: (SafeTimer::timer_thread()+0x1af) [0xa5496f] 19: (SafeTimerThread::entry()+0xd) [0xa55f2d] 20: (()+0x6b50) [0x7f3eb0d70b50] 21: (clone()+0x6d) [0x7f3eaf8127bd] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 10/10 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mds.codetest.log --- end dump of recent events ---