$ sudo rados --debug-ms=20 --debug-objecter=20 --pool rbd rm rbd_data.3d31f5756d622c.0000000000005b87 2017-03-07 21:30:41.379343 7fbba5bdca40 10 -- :/0 ready :/0 2017-03-07 21:30:41.379553 7fbba5bdca40 1 -- :/0 messenger.start 2017-03-07 21:30:41.380482 7fbb87fff700 10 -- :/761686341 reaper_entry start 2017-03-07 21:30:41.380526 7fbb87fff700 10 -- :/761686341 reaper 2017-03-07 21:30:41.380545 7fbb87fff700 10 -- :/761686341 reaper done 2017-03-07 21:30:41.380969 7fbba5bdca40 10 -- :/761686341 connect_rank to 10.21.199.4:6789/0, creating pipe and registering 2017-03-07 21:30:41.381072 7fbba5bdca40 10 -- :/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7fbba669b1b0).register_pipe 2017-03-07 21:30:41.381106 7fbba5bdca40 10 -- :/761686341 get_connection mon.1 10.21.199.4:6789/0 new 0x7fbba6699ef0 2017-03-07 21:30:41.381119 7fbba5bdca40 20 -- :/761686341 send_keepalive con 0x7fbba669b1b0, have pipe. 2017-03-07 21:30:41.381136 7fbba5bdca40 1 -- :/761686341 --> 10.21.199.4:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- ?+0 0x7fbba669c210 con 0x7fbba669b1b0 2017-03-07 21:30:41.381249 7fbba5bd2700 10 -- :/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7fbba669b1b0).writer: state = connecting policy.server=0 2017-03-07 21:30:41.381294 7fbba5bd2700 10 -- :/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7fbba669b1b0).connect 0 2017-03-07 21:30:41.381324 7fbba5bdca40 20 -- :/761686341 submit_message auth(proto 0 30 bytes epoch 0) v1 remote, 10.21.199.4:6789/0, have pipe. 2017-03-07 21:30:41.381335 7fbba5bd2700 10 -- :/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7fbba669b1b0).connecting to 10.21.199.4:6789/0 2017-03-07 21:30:41.382202 7fbba5bd2700 20 -- :/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=1 pgs=0 cs=0 l=1 c=0x7fbba669b1b0).connect read peer addr 10.21.199.4:6789/0 on socket 4 2017-03-07 21:30:41.382275 7fbba5bd2700 20 -- :/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=1 pgs=0 cs=0 l=1 c=0x7fbba669b1b0).connect peer addr for me is 10.21.199.3:55678/0 2017-03-07 21:30:41.382290 7fbba5bd2700 1 -- 10.21.199.3:0/761686341 learned my addr 10.21.199.3:0/761686341 2017-03-07 21:30:41.382329 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=1 pgs=0 cs=0 l=1 c=0x7fbba669b1b0).connect sent my addr 10.21.199.3:0/761686341 2017-03-07 21:30:41.382347 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=1 pgs=0 cs=0 l=1 c=0x7fbba669b1b0).connect sending gseq=1 cseq=0 proto=15 2017-03-07 21:30:41.382368 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=1 pgs=0 cs=0 l=1 c=0x7fbba669b1b0).connect wrote (self +) cseq, waiting for reply 2017-03-07 21:30:41.382605 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=1 pgs=0 cs=0 l=1 c=0x7fbba669b1b0).connect got reply tag 1 connect_seq 1 global_seq 1138536 proto 15 flags 1 features 576460752032874495 2017-03-07 21:30:41.382651 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).connect success 1, lossy = 1, features 576460752032874495 2017-03-07 21:30:41.382686 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).connect starting reader 2017-03-07 21:30:41.382779 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.382790 7fbb8d359700 10 client.?.objecter ms_handle_connect 0x7fbba669b1b0 2017-03-07 21:30:41.382808 7fbb8d359700 10 client.?.objecter resend_mon_ops 2017-03-07 21:30:41.382794 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_keepalive2 14 2017-03-07 21:30:41.382792 2017-03-07 21:30:41.382848 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer encoding 1 features 576460752032874495 0x7fbba669c210 auth(proto 0 30 bytes epoch 0) v1 2017-03-07 21:30:41.382882 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.382910 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer no session security 2017-03-07 21:30:41.382922 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sending 1 0x7fbba669c210 2017-03-07 21:30:41.382966 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.382977 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:41.383171 7fbba599d700 2 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got KEEPALIVE_ACK 2017-03-07 21:30:41.383197 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.383948 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ACK 2017-03-07 21:30:41.383984 7fbba599d700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ack seq 1 2017-03-07 21:30:41.384010 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.384019 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got MSG 2017-03-07 21:30:41.384032 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got envelope type=4 src mon.1 front=800 data=0 off 0 2017-03-07 21:30:41.384049 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader wants 800 from dispatch throttler 0/0 2017-03-07 21:30:41.384064 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got front 800 2017-03-07 21:30:41.384074 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).aborted = 0 2017-03-07 21:30:41.384081 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got 800 + 0 + 0 byte message 2017-03-07 21:30:41.384115 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).No session security set 2017-03-07 21:30:41.384135 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got message 1 0x7fbb70001e40 mon_map magic: 0 v1 2017-03-07 21:30:41.384153 7fbba599d700 20 -- 10.21.199.3:0/761686341 queue 0x7fbb70001e40 prio 196 2017-03-07 21:30:41.384173 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.384181 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got MSG 2017-03-07 21:30:41.384188 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got envelope type=18 src mon.1 front=33 data=0 off 0 2017-03-07 21:30:41.384195 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader wants 33 from dispatch throttler 0/0 2017-03-07 21:30:41.384180 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.384203 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got front 33 2017-03-07 21:30:41.384213 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).aborted = 0 2017-03-07 21:30:41.384220 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got 33 + 0 + 0 byte message 2017-03-07 21:30:41.384216 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_ack 1 2017-03-07 21:30:41.384239 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).No session security set 2017-03-07 21:30:41.384236 7fbb8d359700 1 -- 10.21.199.3:0/761686341 <== mon.1 10.21.199.4:6789/0 1 ==== mon_map magic: 0 v1 ==== 800+0+0 (2252526554 0 0) 0x7fbb70001e40 con 0x7fbba669b1b0 2017-03-07 21:30:41.384249 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got message 2 0x7fbb70002fc0 auth_reply(proto 2 0 (0) Success) v1 2017-03-07 21:30:41.384284 7fbba599d700 20 -- 10.21.199.3:0/761686341 queue 0x7fbb70002fc0 prio 196 2017-03-07 21:30:41.384296 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.384306 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.384323 7fbb8d359700 10 -- 10.21.199.3:0/761686341 dispatch_throttle_release 800 to dispatch throttler 0/0 2017-03-07 21:30:41.384335 7fbb8d359700 20 -- 10.21.199.3:0/761686341 done calling dispatch on 0x7fbb70001e40 2017-03-07 21:30:41.384329 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_ack 2 2017-03-07 21:30:41.384354 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.384365 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:41.384351 7fbb8d359700 1 -- 10.21.199.3:0/761686341 <== mon.1 10.21.199.4:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (955091416 0 0) 0x7fbb70002fc0 con 0x7fbba669b1b0 2017-03-07 21:30:41.384554 7fbb8d359700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.4:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- ?+0 0x7fbb7c001e20 con 0x7fbba669b1b0 2017-03-07 21:30:41.384580 7fbb8d359700 20 -- 10.21.199.3:0/761686341 submit_message auth(proto 2 32 bytes epoch 0) v1 remote, 10.21.199.4:6789/0, have pipe. 2017-03-07 21:30:41.384598 7fbb8d359700 10 -- 10.21.199.3:0/761686341 dispatch_throttle_release 33 to dispatch throttler 0/0 2017-03-07 21:30:41.384603 7fbb8d359700 20 -- 10.21.199.3:0/761686341 done calling dispatch on 0x7fbb70002fc0 2017-03-07 21:30:41.384665 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.384693 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer encoding 2 features 576460752032874495 0x7fbb7c001e20 auth(proto 2 32 bytes epoch 0) v1 2017-03-07 21:30:41.384726 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer no session security 2017-03-07 21:30:41.384739 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sending 2 0x7fbb7c001e20 2017-03-07 21:30:41.384790 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.384801 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:41.385669 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ACK 2017-03-07 21:30:41.385701 7fbba599d700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ack seq 2 2017-03-07 21:30:41.385712 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.385719 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got MSG 2017-03-07 21:30:41.385727 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got envelope type=18 src mon.1 front=206 data=0 off 0 2017-03-07 21:30:41.385737 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader wants 206 from dispatch throttler 0/0 2017-03-07 21:30:41.385748 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got front 206 2017-03-07 21:30:41.385755 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).aborted = 0 2017-03-07 21:30:41.385761 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got 206 + 0 + 0 byte message 2017-03-07 21:30:41.385786 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).No session security set 2017-03-07 21:30:41.385802 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got message 3 0x7fbb70004000 auth_reply(proto 2 0 (0) Success) v1 2017-03-07 21:30:41.385827 7fbba599d700 20 -- 10.21.199.3:0/761686341 queue 0x7fbb70004000 prio 196 2017-03-07 21:30:41.385840 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.385847 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.385874 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_ack 3 2017-03-07 21:30:41.385896 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.385905 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:41.385907 7fbb8d359700 1 -- 10.21.199.3:0/761686341 <== mon.1 10.21.199.4:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (2257616673 0 0) 0x7fbb70004000 con 0x7fbba669b1b0 2017-03-07 21:30:41.386086 7fbb8d359700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.4:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- ?+0 0x7fbb7c0022a0 con 0x7fbba669b1b0 2017-03-07 21:30:41.386100 7fbb8d359700 20 -- 10.21.199.3:0/761686341 submit_message auth(proto 2 165 bytes epoch 0) v1 remote, 10.21.199.4:6789/0, have pipe. 2017-03-07 21:30:41.386116 7fbb8d359700 10 -- 10.21.199.3:0/761686341 dispatch_throttle_release 206 to dispatch throttler 0/0 2017-03-07 21:30:41.386121 7fbb8d359700 20 -- 10.21.199.3:0/761686341 done calling dispatch on 0x7fbb70004000 2017-03-07 21:30:41.386140 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.386175 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer encoding 3 features 576460752032874495 0x7fbb7c0022a0 auth(proto 2 165 bytes epoch 0) v1 2017-03-07 21:30:41.386227 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer no session security 2017-03-07 21:30:41.386250 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sending 3 0x7fbb7c0022a0 2017-03-07 21:30:41.386312 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.386331 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:41.387098 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ACK 2017-03-07 21:30:41.387137 7fbba599d700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ack seq 3 2017-03-07 21:30:41.387153 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.387166 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got MSG 2017-03-07 21:30:41.387175 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got envelope type=18 src mon.1 front=393 data=0 off 0 2017-03-07 21:30:41.387189 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader wants 393 from dispatch throttler 0/0 2017-03-07 21:30:41.387199 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got front 393 2017-03-07 21:30:41.387206 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).aborted = 0 2017-03-07 21:30:41.387222 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got 393 + 0 + 0 byte message 2017-03-07 21:30:41.387238 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).No session security set 2017-03-07 21:30:41.387254 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got message 4 0x7fbb700050f0 auth_reply(proto 2 0 (0) Success) v1 2017-03-07 21:30:41.387278 7fbba599d700 20 -- 10.21.199.3:0/761686341 queue 0x7fbb700050f0 prio 196 2017-03-07 21:30:41.387309 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.387312 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.387347 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_ack 4 2017-03-07 21:30:41.387375 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.387361 7fbb8d359700 1 -- 10.21.199.3:0/761686341 <== mon.1 10.21.199.4:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 393+0+0 (3778660437 0 0) 0x7fbb700050f0 con 0x7fbba669b1b0 2017-03-07 21:30:41.387393 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:41.387509 7fbb8d359700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.4:6789/0 -- mon_subscribe({monmap=0+}) v2 -- ?+0 0x7fbba669ff90 con 0x7fbba669b1b0 2017-03-07 21:30:41.387530 7fbb8d359700 20 -- 10.21.199.3:0/761686341 submit_message mon_subscribe({monmap=0+}) v2 remote, 10.21.199.4:6789/0, have pipe. 2017-03-07 21:30:41.387560 7fbb8d359700 10 -- 10.21.199.3:0/761686341 dispatch_throttle_release 393 to dispatch throttler 0/0 2017-03-07 21:30:41.387566 7fbb8d359700 20 -- 10.21.199.3:0/761686341 done calling dispatch on 0x7fbb700050f0 2017-03-07 21:30:41.387566 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.387602 7fbba5bdca40 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:30:41.387595 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer encoding 4 features 576460752032874495 0x7fbba669ff90 mon_subscribe({monmap=0+}) v2 2017-03-07 21:30:41.387619 7fbba5bdca40 1 -- 10.21.199.3:0/761686341 --> 10.21.199.4:6789/0 -- mon_subscribe({osdmap=0}) v2 -- ?+0 0x7fbba669c210 con 0x7fbba669b1b0 2017-03-07 21:30:41.387640 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer no session security 2017-03-07 21:30:41.387670 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sending 4 0x7fbba669ff90 2017-03-07 21:30:41.387678 7fbba5bdca40 20 -- 10.21.199.3:0/761686341 submit_message mon_subscribe({osdmap=0}) v2 remote, 10.21.199.4:6789/0, have pipe. 2017-03-07 21:30:41.387726 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.387740 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer encoding 5 features 576460752032874495 0x7fbba669c210 mon_subscribe({osdmap=0}) v2 2017-03-07 21:30:41.387755 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer no session security 2017-03-07 21:30:41.387763 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sending 5 0x7fbba669c210 2017-03-07 21:30:41.387791 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.387801 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:41.388127 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ACK 2017-03-07 21:30:41.388192 7fbba599d700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ack seq 4 2017-03-07 21:30:41.388204 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.388212 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ACK 2017-03-07 21:30:41.388218 7fbba599d700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ack seq 5 2017-03-07 21:30:41.388224 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.388229 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got MSG 2017-03-07 21:30:41.388236 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got envelope type=4 src mon.1 front=800 data=0 off 0 2017-03-07 21:30:41.388251 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader wants 800 from dispatch throttler 0/0 2017-03-07 21:30:41.388261 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got front 800 2017-03-07 21:30:41.388267 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).aborted = 0 2017-03-07 21:30:41.388273 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got 800 + 0 + 0 byte message 2017-03-07 21:30:41.388329 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).No session security set 2017-03-07 21:30:41.388348 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got message 5 0x7fbb70006860 mon_map magic: 0 v1 2017-03-07 21:30:41.388427 7fbba599d700 20 -- 10.21.199.3:0/761686341 queue 0x7fbb70006860 prio 196 2017-03-07 21:30:41.388468 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.388499 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got MSG 2017-03-07 21:30:41.388516 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got envelope type=41 src mon.1 front=542252 data=0 off 0 2017-03-07 21:30:41.388508 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.388526 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader wants 542252 from dispatch throttler 0/0 2017-03-07 21:30:41.388535 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_ack 5 2017-03-07 21:30:41.388529 7fbb8d359700 1 -- 10.21.199.3:0/761686341 <== mon.1 10.21.199.4:6789/0 5 ==== mon_map magic: 0 v1 ==== 800+0+0 (2252526554 0 0) 0x7fbb70006860 con 0x7fbba669b1b0 2017-03-07 21:30:41.388577 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.388581 7fbb8d359700 10 -- 10.21.199.3:0/761686341 dispatch_throttle_release 800 to dispatch throttler 0/0 2017-03-07 21:30:41.388588 7fbb8d359700 20 -- 10.21.199.3:0/761686341 done calling dispatch on 0x7fbb70006860 2017-03-07 21:30:41.388587 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:41.390278 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got front 542252 2017-03-07 21:30:41.390311 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).aborted = 0 2017-03-07 21:30:41.390320 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got 542252 + 0 + 0 byte message 2017-03-07 21:30:41.390427 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).No session security set 2017-03-07 21:30:41.390447 7fbba599d700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got message 6 0x7fbb70006260 osd_map(234774..234774 src has 234241..234774) v3 2017-03-07 21:30:41.390464 7fbba599d700 20 -- 10.21.199.3:0/761686341 queue 0x7fbb70006260 prio 196 2017-03-07 21:30:41.390490 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:41.390501 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.390536 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_ack 6 2017-03-07 21:30:41.390561 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:41.390602 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:41.390549 7fbb8d359700 1 -- 10.21.199.3:0/761686341 <== mon.1 10.21.199.4:6789/0 6 ==== osd_map(234774..234774 src has 234241..234774) v3 ==== 542252+0+0 (3963718397 0 0) 0x7fbb70006260 con 0x7fbba669b1b0 2017-03-07 21:30:41.390631 7fbb8d359700 10 client.4113901.objecter ms_dispatch 0x7fbba663df90 osd_map(234774..234774 src has 234241..234774) v3 2017-03-07 21:30:41.390655 7fbb8d359700 3 client.4113901.objecter handle_osd_map got epochs [234774,234774] > 0 2017-03-07 21:30:41.390662 7fbb8d359700 3 client.4113901.objecter handle_osd_map decoding full epoch 234774 2017-03-07 21:30:41.393935 7fbb8d359700 20 client.4113901.objecter dump_active .. 0 homeless 2017-03-07 21:30:41.394002 7fbb8d359700 10 -- 10.21.199.3:0/761686341 dispatch_throttle_release 542252 to dispatch throttler 0/0 2017-03-07 21:30:41.394012 7fbb8d359700 20 -- 10.21.199.3:0/761686341 done calling dispatch on 0x7fbb70006260 2017-03-07 21:30:41.394180 7fbba5bdca40 10 client.4113901.objecter _op_submit op 0x7fbba669f490 2017-03-07 21:30:41.394262 7fbba5bdca40 10 client.4113901.objecter _calc_target pgid 4.bc2dcdb7 acting [751,45,606] 2017-03-07 21:30:41.394282 7fbba5bdca40 10 -- 10.21.199.3:0/761686341 connect_rank to 10.21.199.42:6820/271935, creating pipe and registering 2017-03-07 21:30:41.394359 7fbba5bdca40 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).register_pipe 2017-03-07 21:30:41.394386 7fbba5bdca40 10 -- 10.21.199.3:0/761686341 get_connection osd.751 10.21.199.42:6820/271935 new 0x7fbba669fe70 2017-03-07 21:30:41.394400 7fbba5bdca40 20 client.4113901.objecter _get_session s=0x7fbba669f6d0 osd=751 2 2017-03-07 21:30:41.394404 7fbba5bdca40 20 client.4113901.objecter note: not requesting ack 2017-03-07 21:30:41.394446 7fbba5bdca40 10 client.4113901.objecter _op_submit oid rbd_data.3d31f5756d622c.0000000000005b87 '@4' '@4' [delete] tid 1 osd.751 2017-03-07 21:30:41.394463 7fbba5bdca40 20 client.4113901.objecter get_session s=0x7fbba669f6d0 osd=751 2 2017-03-07 21:30:41.394468 7fbba5bdca40 15 client.4113901.objecter _session_op_assign 751 1 2017-03-07 21:30:41.394471 7fbba5bdca40 15 client.4113901.objecter _send_op 1 to osd.751 2017-03-07 21:30:41.394466 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).writer: state = connecting policy.server=0 2017-03-07 21:30:41.394474 7fbba5bdca40 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- osd_op(client.4113901.0:1 4.bc2dcdb7 rbd_data.3d31f5756d622c.0000000000005b87 [delete] snapc 0=[] ondisk+write+known_if_redirected e234774) v7 -- ?+0 0x7fbba66a2a00 con 0x7fbba66a1130 2017-03-07 21:30:41.394496 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).connect 0 2017-03-07 21:30:41.394524 7fbba5bdca40 20 -- 10.21.199.3:0/761686341 submit_message osd_op(client.4113901.0:1 4.bc2dcdb7 rbd_data.3d31f5756d622c.0000000000005b87 [delete] snapc 0=[] ondisk+write+known_if_redirected e234774) v7 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:30:41.394545 7fbba5bdca40 20 client.4113901.objecter put_session s=0x7fbba669f6d0 osd=751 3 2017-03-07 21:30:41.394539 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :0 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).connecting to 10.21.199.42:6820/271935 2017-03-07 21:30:41.394548 7fbba5bdca40 5 client.4113901.objecter 0 unacked, 1 uncommitted 2017-03-07 21:30:41.395717 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).connect read peer addr 10.21.199.42:6820/271935 on socket 5 2017-03-07 21:30:41.395755 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).connect peer addr for me is 10.21.199.3:49549/0 2017-03-07 21:30:41.395808 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).connect sent my addr 10.21.199.3:0/761686341 2017-03-07 21:30:41.395906 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).connect.authorizer_len=174 protocol=2 2017-03-07 21:30:41.395941 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).connect sending gseq=2 cseq=0 proto=24 2017-03-07 21:30:41.395980 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).connect wrote (self +) cseq, waiting for reply 2017-03-07 21:30:41.396322 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).connect got reply tag 1 connect_seq 1 global_seq 4910629 proto 24 flags 1 features 576460752032874495 2017-03-07 21:30:41.396357 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=1 pgs=0 cs=0 l=1 c=0x7fbba66a1130).reply.authorizer_len=36 2017-03-07 21:30:41.396414 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).connect success 1, lossy = 1, features 576460752032874495 2017-03-07 21:30:41.396442 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).connect starting reader 2017-03-07 21:30:41.396506 7fbb8d359700 10 client.4113901.objecter ms_handle_connect 0x7fbba66a1130 2017-03-07 21:30:41.396529 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:30:41.396544 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 1 features 576460752032874495 0x7fbba66a2a00 osd_op(client.4113901.0:1 4.bc2dcdb7 rbd_data.3d31f5756d622c.0000000000005b87 [delete] snapc 0=[] ondisk+write+known_if_redirected e234774) v7 2017-03-07 21:30:41.396593 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 1): sig = 0 2017-03-07 21:30:41.396611 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 1 0x7fbba66a2a00 2017-03-07 21:30:41.396662 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:30:41.396674 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:30:41.396750 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:30:41.599631 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:30:41.599671 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 1 2017-03-07 21:30:41.599681 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:30:44.381117 7fbb877fe700 20 -- 10.21.199.3:0/761686341 send_keepalive con 0x7fbba669b1b0, have pipe. 2017-03-07 21:30:44.381232 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:44.381268 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_keepalive2 14 2017-03-07 21:30:44.381266 2017-03-07 21:30:44.381326 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:44.381357 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:44.381655 7fbba599d700 2 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got KEEPALIVE_ACK 2017-03-07 21:30:44.381719 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:46.387803 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:30:51.387968 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:30:54.381305 7fbb877fe700 20 -- 10.21.199.3:0/761686341 send_keepalive con 0x7fbba669b1b0, have pipe. 2017-03-07 21:30:54.381412 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:54.381449 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_keepalive2 14 2017-03-07 21:30:54.381447 2017-03-07 21:30:54.381509 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:54.381540 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:54.381823 7fbba599d700 2 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got KEEPALIVE_ACK 2017-03-07 21:30:54.381891 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:30:56.388097 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:30:56.388133 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:30:56.388144 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:30:56.388175 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.4:6789/0 -- mon_subscribe({osdmap=234775}) v2 -- ?+0 0x7fbb60000e10 con 0x7fbba669b1b0 2017-03-07 21:30:56.388210 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message mon_subscribe({osdmap=234775}) v2 remote, 10.21.199.4:6789/0, have pipe. 2017-03-07 21:30:56.388233 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60001410 con 0x7fbba66a1130 2017-03-07 21:30:56.388243 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:30:56.388282 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:56.388283 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:30:56.388324 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 2 features 576460752032874495 0x7fbb60001410 ping magic: 0 v1 2017-03-07 21:30:56.388335 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer encoding 6 features 576460752032874495 0x7fbb60000e10 mon_subscribe({osdmap=234775}) v2 2017-03-07 21:30:56.388344 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 2): sig = 0 2017-03-07 21:30:56.388361 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 2 0x7fbb60001410 2017-03-07 21:30:56.388372 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer no session security 2017-03-07 21:30:56.388390 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sending 6 0x7fbb60000e10 2017-03-07 21:30:56.388440 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:30:56.388457 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:30:56.388439 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:30:56.388491 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:30:56.595668 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:30:56.595728 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 2 2017-03-07 21:30:56.595740 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:30:56.596027 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ACK 2017-03-07 21:30:56.596060 7fbba599d700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got ack seq 6 2017-03-07 21:30:56.596069 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:31:01.388349 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:01.388384 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:01.388394 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:01.388410 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60001960 con 0x7fbba66a1130 2017-03-07 21:31:01.388438 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:01.388506 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:01.388538 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 3 features 576460752032874495 0x7fbb60001960 ping magic: 0 v1 2017-03-07 21:31:01.388552 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 3): sig = 0 2017-03-07 21:31:01.388565 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 3 0x7fbb60001960 2017-03-07 21:31:01.388628 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:01.388676 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:01.595686 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:01.595753 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 3 2017-03-07 21:31:01.595764 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:31:04.381471 7fbb877fe700 20 -- 10.21.199.3:0/761686341 send_keepalive con 0x7fbba669b1b0, have pipe. 2017-03-07 21:31:04.381564 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:04.381600 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_keepalive2 14 2017-03-07 21:31:04.381598 2017-03-07 21:31:04.381657 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:04.381686 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:31:04.382059 7fbba599d700 2 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got KEEPALIVE_ACK 2017-03-07 21:31:04.382130 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:31:06.388566 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:06.388607 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:06.388640 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:06.388656 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60002020 con 0x7fbba66a1130 2017-03-07 21:31:06.388681 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:06.388751 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:06.388783 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 4 features 576460752032874495 0x7fbb60002020 ping magic: 0 v1 2017-03-07 21:31:06.388797 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 4): sig = 0 2017-03-07 21:31:06.388810 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 4 0x7fbb60002020 2017-03-07 21:31:06.388873 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:06.388922 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:06.591659 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:06.591734 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 4 2017-03-07 21:31:06.591749 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:31:11.388846 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:11.388884 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:11.388895 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:11.388912 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb600025e0 con 0x7fbba66a1130 2017-03-07 21:31:11.388941 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:11.389014 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:11.389045 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 5 features 576460752032874495 0x7fbb600025e0 ping magic: 0 v1 2017-03-07 21:31:11.389059 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 5): sig = 0 2017-03-07 21:31:11.389116 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 5 0x7fbb600025e0 2017-03-07 21:31:11.389168 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:11.389214 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:11.595695 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:11.595761 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 5 2017-03-07 21:31:11.595773 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:31:14.381621 7fbb877fe700 20 -- 10.21.199.3:0/761686341 send_keepalive con 0x7fbba669b1b0, have pipe. 2017-03-07 21:31:14.381688 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:14.381725 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_keepalive2 14 2017-03-07 21:31:14.381723 2017-03-07 21:31:14.381770 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:14.381787 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:31:14.382069 7fbba599d700 2 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got KEEPALIVE_ACK 2017-03-07 21:31:14.382140 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:31:16.389060 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:16.389101 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:16.389113 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:16.389132 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60002c90 con 0x7fbba66a1130 2017-03-07 21:31:16.389183 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:16.389229 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:16.389269 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 6 features 576460752032874495 0x7fbb60002c90 ping magic: 0 v1 2017-03-07 21:31:16.389302 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 6): sig = 0 2017-03-07 21:31:16.389319 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 6 0x7fbb60002c90 2017-03-07 21:31:16.389395 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:16.389454 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:16.591668 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:16.591737 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 6 2017-03-07 21:31:16.591764 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:31:21.389323 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:21.389367 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:21.389378 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:21.389396 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60003250 con 0x7fbba66a1130 2017-03-07 21:31:21.389425 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:21.389493 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:21.389525 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 7 features 576460752032874495 0x7fbb60003250 ping magic: 0 v1 2017-03-07 21:31:21.389539 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 7): sig = 0 2017-03-07 21:31:21.389552 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 7 0x7fbb60003250 2017-03-07 21:31:21.389615 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:21.389638 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:21.591736 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:21.591773 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 7 2017-03-07 21:31:21.591784 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:31:24.381784 7fbb877fe700 20 -- 10.21.199.3:0/761686341 send_keepalive con 0x7fbba669b1b0, have pipe. 2017-03-07 21:31:24.381887 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:24.381922 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_keepalive2 14 2017-03-07 21:31:24.381921 2017-03-07 21:31:24.381982 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:24.382012 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:31:24.382305 7fbba599d700 2 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got KEEPALIVE_ACK 2017-03-07 21:31:24.382369 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:31:26.389544 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:26.389583 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:26.389594 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:26.389610 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb600038f0 con 0x7fbba66a1130 2017-03-07 21:31:26.389634 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:26.389671 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:26.389709 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 8 features 576460752032874495 0x7fbb600038f0 ping magic: 0 v1 2017-03-07 21:31:26.389744 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 8): sig = 0 2017-03-07 21:31:26.389768 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 8 0x7fbb600038f0 2017-03-07 21:31:26.389851 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:26.389872 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:26.591698 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:26.591755 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 8 2017-03-07 21:31:26.591771 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:31:31.389761 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:31.389798 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:31.389810 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:31.389839 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60003e90 con 0x7fbba66a1130 2017-03-07 21:31:31.389868 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:31.389904 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:31.389943 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 9 features 576460752032874495 0x7fbb60003e90 ping magic: 0 v1 2017-03-07 21:31:31.389964 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 9): sig = 0 2017-03-07 21:31:31.389985 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 9 0x7fbb60003e90 2017-03-07 21:31:31.390099 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:31.390120 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:31.591681 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:31.591722 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 9 2017-03-07 21:31:31.591739 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:31:34.381973 7fbb877fe700 20 -- 10.21.199.3:0/761686341 send_keepalive con 0x7fbba669b1b0, have pipe. 2017-03-07 21:31:34.382096 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:34.382133 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_keepalive2 14 2017-03-07 21:31:34.382131 2017-03-07 21:31:34.382224 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:34.382247 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:31:34.382544 7fbba599d700 2 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got KEEPALIVE_ACK 2017-03-07 21:31:34.382612 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:31:36.389975 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:36.390016 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:36.390027 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:36.390043 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60004430 con 0x7fbba66a1130 2017-03-07 21:31:36.390071 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:36.390142 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:36.390280 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 10 features 576460752032874495 0x7fbb60004430 ping magic: 0 v1 2017-03-07 21:31:36.390308 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 10): sig = 0 2017-03-07 21:31:36.390321 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 10 0x7fbb60004430 2017-03-07 21:31:36.390366 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:36.390395 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:36.591629 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:36.591691 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 10 2017-03-07 21:31:36.591703 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:31:41.390255 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:41.390290 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:41.390301 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:41.390317 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb600049d0 con 0x7fbba66a1130 2017-03-07 21:31:41.390345 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:41.390388 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:41.390421 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 11 features 576460752032874495 0x7fbb600049d0 ping magic: 0 v1 2017-03-07 21:31:41.390435 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 11): sig = 0 2017-03-07 21:31:41.390448 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 11 0x7fbb600049d0 2017-03-07 21:31:41.390505 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:41.390540 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:41.591682 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:41.591719 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 11 2017-03-07 21:31:41.591730 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:31:44.382172 7fbb877fe700 20 -- 10.21.199.3:0/761686341 send_keepalive con 0x7fbba669b1b0, have pipe. 2017-03-07 21:31:44.382290 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:44.382326 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_keepalive2 14 2017-03-07 21:31:44.382325 2017-03-07 21:31:44.382400 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:44.382439 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:31:44.382758 7fbba599d700 2 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got KEEPALIVE_ACK 2017-03-07 21:31:44.382828 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:31:46.390484 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:46.390524 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:46.390535 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:46.390566 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60004f70 con 0x7fbba66a1130 2017-03-07 21:31:46.390595 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:46.390661 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:46.390704 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 12 features 576460752032874495 0x7fbb60004f70 ping magic: 0 v1 2017-03-07 21:31:46.390719 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 12): sig = 0 2017-03-07 21:31:46.390731 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 12 0x7fbb60004f70 2017-03-07 21:31:46.390794 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:46.390816 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:46.591707 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:46.591772 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 12 2017-03-07 21:31:46.591783 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:31:51.390725 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:51.390765 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:51.390776 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:51.390792 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60005500 con 0x7fbba66a1130 2017-03-07 21:31:51.390821 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:51.390890 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:51.390922 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 13 features 576460752032874495 0x7fbb60005500 ping magic: 0 v1 2017-03-07 21:31:51.390936 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 13): sig = 0 2017-03-07 21:31:51.390949 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 13 0x7fbb60005500 2017-03-07 21:31:51.391012 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:51.391024 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:51.591679 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:51.591719 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 13 2017-03-07 21:31:51.591729 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:31:54.382365 7fbb877fe700 20 -- 10.21.199.3:0/761686341 send_keepalive con 0x7fbba669b1b0, have pipe. 2017-03-07 21:31:54.382469 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:54.382504 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_keepalive2 14 2017-03-07 21:31:54.382502 2017-03-07 21:31:54.382560 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:31:54.382571 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:31:54.382880 7fbba599d700 2 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got KEEPALIVE_ACK 2017-03-07 21:31:54.382944 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:31:56.390941 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:31:56.390977 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:31:56.390988 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:31:56.391004 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60005a80 con 0x7fbba66a1130 2017-03-07 21:31:56.391027 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:31:56.391100 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:56.391145 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 14 features 576460752032874495 0x7fbb60005a80 ping magic: 0 v1 2017-03-07 21:31:56.391159 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 14): sig = 0 2017-03-07 21:31:56.391172 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 14 0x7fbb60005a80 2017-03-07 21:31:56.391238 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:31:56.391250 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:31:56.591661 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:31:56.591720 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 14 2017-03-07 21:31:56.591740 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:32:01.391154 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:32:01.391192 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:32:01.391203 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:32:01.391232 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60006000 con 0x7fbba66a1130 2017-03-07 21:32:01.391261 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:32:01.391332 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:32:01.391365 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 15 features 576460752032874495 0x7fbb60006000 ping magic: 0 v1 2017-03-07 21:32:01.391379 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 15): sig = 0 2017-03-07 21:32:01.391392 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 15 0x7fbb60006000 2017-03-07 21:32:01.391455 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:32:01.391504 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:32:01.591719 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:32:01.591797 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 15 2017-03-07 21:32:01.591809 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:32:04.382526 7fbb877fe700 20 -- 10.21.199.3:0/761686341 send_keepalive con 0x7fbba669b1b0, have pipe. 2017-03-07 21:32:04.382611 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:32:04.382649 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).write_keepalive2 14 2017-03-07 21:32:04.382647 2017-03-07 21:32:04.382708 7fbba5bd2700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer: state = open policy.server=0 2017-03-07 21:32:04.382719 7fbba5bd2700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).writer sleeping 2017-03-07 21:32:04.383073 7fbba599d700 2 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader got KEEPALIVE_ACK 2017-03-07 21:32:04.383155 7fbba599d700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.4:6789/0 pipe(0x7fbba6699ef0 sd=4 :55678 s=2 pgs=1138536 cs=1 l=1 c=0x7fbba669b1b0).reader reading tag... 2017-03-07 21:32:06.391427 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:32:06.391467 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:32:06.391478 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:32:06.391494 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60006580 con 0x7fbba66a1130 2017-03-07 21:32:06.391526 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:32:06.391566 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:32:06.391609 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 16 features 576460752032874495 0x7fbb60006580 ping magic: 0 v1 2017-03-07 21:32:06.391624 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 16): sig = 0 2017-03-07 21:32:06.391636 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 16 0x7fbb60006580 2017-03-07 21:32:06.391697 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:32:06.391709 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:32:06.595723 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:32:06.595790 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 16 2017-03-07 21:32:06.595801 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag... 2017-03-07 21:32:11.391664 7fbb8db5a700 10 client.4113901.objecter tick 2017-03-07 21:32:11.391701 7fbb8db5a700 2 client.4113901.objecter tid 1 on osd.751 is laggy 2017-03-07 21:32:11.391712 7fbb8db5a700 10 client.4113901.objecter _maybe_request_map subscribing (onetime) to next osd map 2017-03-07 21:32:11.391728 7fbb8db5a700 1 -- 10.21.199.3:0/761686341 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7fbb60006b00 con 0x7fbba66a1130 2017-03-07 21:32:11.391756 7fbb8db5a700 20 -- 10.21.199.3:0/761686341 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe. 2017-03-07 21:32:11.391792 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:32:11.391821 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer encoding 17 features 576460752032874495 0x7fbb60006b00 ping magic: 0 v1 2017-03-07 21:32:11.391835 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer signed seq # 17): sig = 0 2017-03-07 21:32:11.391847 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sending 17 0x7fbb60006b00 2017-03-07 21:32:11.391909 7fbb8c357700 10 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer: state = open policy.server=0 2017-03-07 21:32:11.391971 7fbb8c357700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).writer sleeping 2017-03-07 21:32:11.595709 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ACK 2017-03-07 21:32:11.595749 7fbb8c256700 15 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader got ack seq 17 2017-03-07 21:32:11.595760 7fbb8c256700 20 -- 10.21.199.3:0/761686341 >> 10.21.199.42:6820/271935 pipe(0x7fbba669fe70 sd=5 :49549 s=2 pgs=4910629 cs=1 l=1 c=0x7fbba66a1130).reader reading tag...