$ sudo rados --debug-ms=20 --debug-objecter=20 --pool rbd rm rbd_data.3d31f5756d622c.0000000000005b87
2017-03-07 21:34:20.458132 7f571bffaa40 10 -- :/0 ready :/0
2017-03-07 21:34:20.458337 7f571bffaa40  1 -- :/0 messenger.start
2017-03-07 21:34:20.458527 7f5702775700 10 -- :/3239701293 reaper_entry start
2017-03-07 21:34:20.458551 7f5702775700 10 -- :/3239701293 reaper
2017-03-07 21:34:20.458559 7f5702775700 10 -- :/3239701293 reaper done
2017-03-07 21:34:20.458961 7f571bffaa40 10 -- :/3239701293 connect_rank to 10.21.199.5:6789/0, creating pipe and registering
2017-03-07 21:34:20.459056 7f571bffaa40 10 -- :/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7f571cf191b0).register_pipe
2017-03-07 21:34:20.459083 7f571bffaa40 10 -- :/3239701293 get_connection mon.2 10.21.199.5:6789/0 new 0x7f571cf17ef0
2017-03-07 21:34:20.459094 7f571bffaa40 20 -- :/3239701293 send_keepalive con 0x7f571cf191b0, have pipe.
2017-03-07 21:34:20.459109 7f571bffaa40  1 -- :/3239701293 --> 10.21.199.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- ?+0 0x7f571cf1a290 con 0x7f571cf191b0
2017-03-07 21:34:20.459153 7f571bff0700 10 -- :/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7f571cf191b0).writer: state = connecting policy.server=0
2017-03-07 21:34:20.459207 7f571bff0700 10 -- :/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7f571cf191b0).connect 0
2017-03-07 21:34:20.459253 7f571bff0700 10 -- :/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f571cf191b0).connecting to 10.21.199.5:6789/0
2017-03-07 21:34:20.459284 7f571bffaa40 20 -- :/3239701293 submit_message auth(proto 0 30 bytes epoch 0) v1 remote, 10.21.199.5:6789/0, have pipe.
2017-03-07 21:34:20.460362 7f571bff0700 20 -- :/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=1 pgs=0 cs=0 l=1 c=0x7f571cf191b0).connect read peer addr 10.21.199.5:6789/0 on socket 4
2017-03-07 21:34:20.460407 7f571bff0700 20 -- :/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=1 pgs=0 cs=0 l=1 c=0x7f571cf191b0).connect peer addr for me is 10.21.199.3:56520/0
2017-03-07 21:34:20.460420 7f571bff0700  1 -- 10.21.199.3:0/3239701293 learned my addr 10.21.199.3:0/3239701293
2017-03-07 21:34:20.460466 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=1 pgs=0 cs=0 l=1 c=0x7f571cf191b0).connect sent my addr 10.21.199.3:0/3239701293
2017-03-07 21:34:20.460483 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=1 pgs=0 cs=0 l=1 c=0x7f571cf191b0).connect sending gseq=1 cseq=0 proto=15
2017-03-07 21:34:20.460503 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=1 pgs=0 cs=0 l=1 c=0x7f571cf191b0).connect wrote (self +) cseq, waiting for reply
2017-03-07 21:34:20.460681 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=1 pgs=0 cs=0 l=1 c=0x7f571cf191b0).connect got reply tag 1 connect_seq 1 global_seq 819397 proto 15 flags 1 features 576460752032874495
2017-03-07 21:34:20.460722 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).connect success 1, lossy = 1, features 576460752032874495
2017-03-07 21:34:20.460755 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).connect starting reader
2017-03-07 21:34:20.460831 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.460860 7f5703777700 10 client.?.objecter ms_handle_connect 0x7f571cf191b0
2017-03-07 21:34:20.460857 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).write_keepalive2 14 2017-03-07 21:34:20.460855
2017-03-07 21:34:20.460876 7f5703777700 10 client.?.objecter resend_mon_ops
2017-03-07 21:34:20.460906 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer encoding 1 features 576460752032874495 0x7f571cf1a290 auth(proto 0 30 bytes epoch 0) v1
2017-03-07 21:34:20.460925 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.460947 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer no session security
2017-03-07 21:34:20.460966 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sending 1 0x7f571cf1a290
2017-03-07 21:34:20.461001 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.461011 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:20.461141 7f571bdbb700  2 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got KEEPALIVE_ACK
2017-03-07 21:34:20.461166 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.461395 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got ACK
2017-03-07 21:34:20.461428 7f571bdbb700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got ack seq 1
2017-03-07 21:34:20.461447 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.461455 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got MSG
2017-03-07 21:34:20.461467 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got envelope type=4 src mon.2 front=800 data=0 off 0
2017-03-07 21:34:20.461485 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader wants 800 from dispatch throttler 0/0
2017-03-07 21:34:20.461499 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got front 800
2017-03-07 21:34:20.461508 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).aborted = 0
2017-03-07 21:34:20.461516 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got 800 + 0 + 0 byte message
2017-03-07 21:34:20.461548 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).No session security set
2017-03-07 21:34:20.461566 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got message 1 0x7f56f0001e40 mon_map magic: 0 v1
2017-03-07 21:34:20.461582 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 queue 0x7f56f0001e40 prio 196
2017-03-07 21:34:20.461596 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.461604 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got MSG
2017-03-07 21:34:20.461610 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got envelope type=18 src mon.2 front=33 data=0 off 0
2017-03-07 21:34:20.461618 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader wants 33 from dispatch throttler 0/0
2017-03-07 21:34:20.461625 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got front 33
2017-03-07 21:34:20.461613 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.461632 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).aborted = 0
2017-03-07 21:34:20.461638 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got 33 + 0 + 0 byte message
2017-03-07 21:34:20.461641 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).write_ack 1
2017-03-07 21:34:20.461656 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).No session security set
2017-03-07 21:34:20.461662 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.461626 7f5703777700  1 -- 10.21.199.3:0/3239701293 <== mon.2 10.21.199.5:6789/0 1 ==== mon_map magic: 0 v1 ==== 800+0+0 (2252526554 0 0) 0x7f56f0001e40 con 0x7f571cf191b0
2017-03-07 21:34:20.461670 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:20.461718 7f5703777700 10 -- 10.21.199.3:0/3239701293 dispatch_throttle_release 800 to dispatch throttler 0/0
2017-03-07 21:34:20.461731 7f5703777700 20 -- 10.21.199.3:0/3239701293 done calling dispatch on 0x7f56f0001e40
2017-03-07 21:34:20.461712 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got message 2 0x7f56f0002fc0 auth_reply(proto 2 0 (0) Success) v1
2017-03-07 21:34:20.461752 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 queue 0x7f56f0002fc0 prio 196
2017-03-07 21:34:20.461768 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.461776 7f5703777700  1 -- 10.21.199.3:0/3239701293 <== mon.2 10.21.199.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (1519945733 0 0) 0x7f56f0002fc0 con 0x7f571cf191b0
2017-03-07 21:34:20.461815 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.461843 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).write_ack 2
2017-03-07 21:34:20.461861 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.461869 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:20.461956 7f5703777700  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.5:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- ?+0 0x7f56ec001e20 con 0x7f571cf191b0
2017-03-07 21:34:20.461971 7f5703777700 20 -- 10.21.199.3:0/3239701293 submit_message auth(proto 2 32 bytes epoch 0) v1 remote, 10.21.199.5:6789/0, have pipe.
2017-03-07 21:34:20.461989 7f5703777700 10 -- 10.21.199.3:0/3239701293 dispatch_throttle_release 33 to dispatch throttler 0/0
2017-03-07 21:34:20.461994 7f5703777700 20 -- 10.21.199.3:0/3239701293 done calling dispatch on 0x7f56f0002fc0
2017-03-07 21:34:20.462038 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.462065 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer encoding 2 features 576460752032874495 0x7f56ec001e20 auth(proto 2 32 bytes epoch 0) v1
2017-03-07 21:34:20.462097 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer no session security
2017-03-07 21:34:20.462110 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sending 2 0x7f56ec001e20
2017-03-07 21:34:20.462159 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.462170 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:20.462871 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got ACK
2017-03-07 21:34:20.462903 7f571bdbb700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got ack seq 2
2017-03-07 21:34:20.462912 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.462919 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got MSG
2017-03-07 21:34:20.462926 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got envelope type=18 src mon.2 front=206 data=0 off 0
2017-03-07 21:34:20.462935 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader wants 206 from dispatch throttler 0/0
2017-03-07 21:34:20.462945 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got front 206
2017-03-07 21:34:20.462952 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).aborted = 0
2017-03-07 21:34:20.462958 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got 206 + 0 + 0 byte message
2017-03-07 21:34:20.462983 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).No session security set
2017-03-07 21:34:20.462998 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got message 3 0x7f56f0003ed0 auth_reply(proto 2 0 (0) Success) v1
2017-03-07 21:34:20.463021 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 queue 0x7f56f0003ed0 prio 196
2017-03-07 21:34:20.463034 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.463044 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.463049 7f5703777700  1 -- 10.21.199.3:0/3239701293 <== mon.2 10.21.199.5:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (2386779772 0 0) 0x7f56f0003ed0 con 0x7f571cf191b0
2017-03-07 21:34:20.463071 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).write_ack 3
2017-03-07 21:34:20.463092 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.463099 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:20.463215 7f5703777700  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.5:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- ?+0 0x7f56ec0022a0 con 0x7f571cf191b0
2017-03-07 21:34:20.463233 7f5703777700 20 -- 10.21.199.3:0/3239701293 submit_message auth(proto 2 165 bytes epoch 0) v1 remote, 10.21.199.5:6789/0, have pipe.
2017-03-07 21:34:20.463251 7f5703777700 10 -- 10.21.199.3:0/3239701293 dispatch_throttle_release 206 to dispatch throttler 0/0
2017-03-07 21:34:20.463260 7f5703777700 20 -- 10.21.199.3:0/3239701293 done calling dispatch on 0x7f56f0003ed0
2017-03-07 21:34:20.463299 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.463326 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer encoding 3 features 576460752032874495 0x7f56ec0022a0 auth(proto 2 165 bytes epoch 0) v1
2017-03-07 21:34:20.463363 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer no session security
2017-03-07 21:34:20.463377 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sending 3 0x7f56ec0022a0
2017-03-07 21:34:20.463432 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.463443 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:20.464020 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got ACK
2017-03-07 21:34:20.464043 7f571bdbb700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got ack seq 3
2017-03-07 21:34:20.464050 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.464056 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got MSG
2017-03-07 21:34:20.464063 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got envelope type=18 src mon.2 front=393 data=0 off 0
2017-03-07 21:34:20.464071 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader wants 393 from dispatch throttler 0/0
2017-03-07 21:34:20.464080 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got front 393
2017-03-07 21:34:20.464086 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).aborted = 0
2017-03-07 21:34:20.464092 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got 393 + 0 + 0 byte message
2017-03-07 21:34:20.464113 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).No session security set
2017-03-07 21:34:20.464127 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got message 4 0x7f56f0004fc0 auth_reply(proto 2 0 (0) Success) v1
2017-03-07 21:34:20.464146 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 queue 0x7f56f0004fc0 prio 196
2017-03-07 21:34:20.464157 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.464177 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.464203 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).write_ack 4
2017-03-07 21:34:20.464193 7f5703777700  1 -- 10.21.199.3:0/3239701293 <== mon.2 10.21.199.5:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 393+0+0 (3665620212 0 0) 0x7f56f0004fc0 con 0x7f571cf191b0
2017-03-07 21:34:20.464224 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.464232 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:20.464363 7f5703777700  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.5:6789/0 -- mon_subscribe({monmap=0+}) v2 -- ?+0 0x7f571cf1df90 con 0x7f571cf191b0
2017-03-07 21:34:20.464388 7f5703777700 20 -- 10.21.199.3:0/3239701293 submit_message mon_subscribe({monmap=0+}) v2 remote, 10.21.199.5:6789/0, have pipe.
2017-03-07 21:34:20.464420 7f5703777700 10 -- 10.21.199.3:0/3239701293 dispatch_throttle_release 393 to dispatch throttler 0/0
2017-03-07 21:34:20.464427 7f5703777700 20 -- 10.21.199.3:0/3239701293 done calling dispatch on 0x7f56f0004fc0
2017-03-07 21:34:20.464458 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.464485 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer encoding 4 features 576460752032874495 0x7f571cf1df90 mon_subscribe({monmap=0+}) v2
2017-03-07 21:34:20.464515 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer no session security
2017-03-07 21:34:20.464518 7f571bffaa40 10 client.4094747.objecter _maybe_request_map subscribing (onetime) to next osd map
2017-03-07 21:34:20.464528 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sending 4 0x7f571cf1df90
2017-03-07 21:34:20.464539 7f571bffaa40  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.5:6789/0 -- mon_subscribe({osdmap=0}) v2 -- ?+0 0x7f571cf14000 con 0x7f571cf191b0
2017-03-07 21:34:20.464556 7f571bffaa40 20 -- 10.21.199.3:0/3239701293 submit_message mon_subscribe({osdmap=0}) v2 remote, 10.21.199.5:6789/0, have pipe.
2017-03-07 21:34:20.464578 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.464590 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer encoding 5 features 576460752032874495 0x7f571cf14000 mon_subscribe({osdmap=0}) v2
2017-03-07 21:34:20.464625 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer no session security
2017-03-07 21:34:20.464635 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sending 5 0x7f571cf14000
2017-03-07 21:34:20.464668 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.464677 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:20.464894 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got ACK
2017-03-07 21:34:20.464916 7f571bdbb700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got ack seq 5
2017-03-07 21:34:20.464924 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.464930 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got MSG
2017-03-07 21:34:20.465016 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got envelope type=4 src mon.2 front=800 data=0 off 0
2017-03-07 21:34:20.465028 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader wants 800 from dispatch throttler 0/0
2017-03-07 21:34:20.465126 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got front 800
2017-03-07 21:34:20.465135 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).aborted = 0
2017-03-07 21:34:20.465142 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got 800 + 0 + 0 byte message
2017-03-07 21:34:20.465155 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).No session security set
2017-03-07 21:34:20.465169 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got message 5 0x7f56f0006380 mon_map magic: 0 v1
2017-03-07 21:34:20.465210 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 queue 0x7f56f0006380 prio 196
2017-03-07 21:34:20.465230 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.465239 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got MSG
2017-03-07 21:34:20.465246 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got envelope type=41 src mon.2 front=542252 data=0 off 0
2017-03-07 21:34:20.465238 7f5703777700  1 -- 10.21.199.3:0/3239701293 <== mon.2 10.21.199.5:6789/0 5 ==== mon_map magic: 0 v1 ==== 800+0+0 (2252526554 0 0) 0x7f56f0006380 con 0x7f571cf191b0
2017-03-07 21:34:20.465253 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader wants 542252 from dispatch throttler 0/0
2017-03-07 21:34:20.465238 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.465265 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).write_ack 5
2017-03-07 21:34:20.465289 7f5703777700 10 -- 10.21.199.3:0/3239701293 dispatch_throttle_release 800 to dispatch throttler 0/0
2017-03-07 21:34:20.465290 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.465299 7f5703777700 20 -- 10.21.199.3:0/3239701293 done calling dispatch on 0x7f56f0006380
2017-03-07 21:34:20.465304 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:20.466828 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got front 542252
2017-03-07 21:34:20.466852 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).aborted = 0
2017-03-07 21:34:20.466860 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got 542252 + 0 + 0 byte message
2017-03-07 21:34:20.466966 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).No session security set
2017-03-07 21:34:20.466994 7f571bdbb700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got message 6 0x7f56f0005d80 osd_map(234774..234774 src has 234241..234774) v3
2017-03-07 21:34:20.467010 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 queue 0x7f56f0005d80 prio 196
2017-03-07 21:34:20.467026 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:20.467034 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.467040 7f5703777700  1 -- 10.21.199.3:0/3239701293 <== mon.2 10.21.199.5:6789/0 6 ==== osd_map(234774..234774 src has 234241..234774) v3 ==== 542252+0+0 (3963718397 0 0) 0x7f56f0005d80 con 0x7f571cf191b0
2017-03-07 21:34:20.467057 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).write_ack 6
2017-03-07 21:34:20.467068 7f5703777700 10 client.4094747.objecter ms_dispatch 0x7f571cebbf90 osd_map(234774..234774 src has 234241..234774) v3
2017-03-07 21:34:20.467081 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:20.467094 7f5703777700  3 client.4094747.objecter handle_osd_map got epochs [234774,234774] > 0
2017-03-07 21:34:20.467110 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:20.467119 7f5703777700  3 client.4094747.objecter handle_osd_map decoding full epoch 234774
2017-03-07 21:34:20.470391 7f5703777700 20 client.4094747.objecter dump_active .. 0 homeless
2017-03-07 21:34:20.470455 7f5703777700 10 -- 10.21.199.3:0/3239701293 dispatch_throttle_release 542252 to dispatch throttler 0/0
2017-03-07 21:34:20.470470 7f5703777700 20 -- 10.21.199.3:0/3239701293 done calling dispatch on 0x7f56f0005d80
2017-03-07 21:34:20.470614 7f571bffaa40 10 client.4094747.objecter _op_submit op 0x7f571cf1d310
2017-03-07 21:34:20.470690 7f571bffaa40 10 client.4094747.objecter _calc_target  pgid 4.bc2dcdb7 acting [751,45,606]
2017-03-07 21:34:20.470712 7f571bffaa40 10 -- 10.21.199.3:0/3239701293 connect_rank to 10.21.199.42:6820/271935, creating pipe and registering
2017-03-07 21:34:20.470796 7f571bffaa40 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).register_pipe
2017-03-07 21:34:20.470815 7f571bffaa40 10 -- 10.21.199.3:0/3239701293 get_connection osd.751 10.21.199.42:6820/271935 new 0x7f571cf1dee0
2017-03-07 21:34:20.470828 7f571bffaa40 20 client.4094747.objecter _get_session s=0x7f571cf1d640 osd=751 2
2017-03-07 21:34:20.470832 7f571bffaa40 20 client.4094747.objecter  note: not requesting ack
2017-03-07 21:34:20.470845 7f571bffaa40 10 client.4094747.objecter _op_submit oid rbd_data.3d31f5756d622c.0000000000005b87 '@4' '@4' [delete] tid 1 osd.751
2017-03-07 21:34:20.470896 7f571bffaa40 20 client.4094747.objecter get_session s=0x7f571cf1d640 osd=751 2
2017-03-07 21:34:20.470902 7f571bffaa40 15 client.4094747.objecter _session_op_assign 751 1
2017-03-07 21:34:20.470905 7f571bffaa40 15 client.4094747.objecter _send_op 1 to osd.751
2017-03-07 21:34:20.470907 7f571bffaa40  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.42:6820/271935 -- osd_op(client.4094747.0:1 4.bc2dcdb7 rbd_data.3d31f5756d622c.0000000000005b87 [delete] snapc 0=[] ondisk+write+known_if_redirected e234774) v7 -- ?+0 0x7f571cf20a70 con 0x7f571cf1f1a0
2017-03-07 21:34:20.470974 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).writer: state = connecting policy.server=0
2017-03-07 21:34:20.471005 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=-1 :0 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).connect 0
2017-03-07 21:34:20.471039 7f571bffaa40 20 -- 10.21.199.3:0/3239701293 submit_message osd_op(client.4094747.0:1 4.bc2dcdb7 rbd_data.3d31f5756d622c.0000000000005b87 [delete] snapc 0=[] ondisk+write+known_if_redirected e234774) v7 remote, 10.21.199.42:6820/271935, have pipe.
2017-03-07 21:34:20.471046 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :0 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).connecting to 10.21.199.42:6820/271935
2017-03-07 21:34:20.471064 7f571bffaa40 20 client.4094747.objecter put_session s=0x7f571cf1d640 osd=751 3
2017-03-07 21:34:20.471068 7f571bffaa40  5 client.4094747.objecter 0 unacked, 1 uncommitted
2017-03-07 21:34:20.472053 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).connect read peer addr 10.21.199.42:6820/271935 on socket 5
2017-03-07 21:34:20.472083 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).connect peer addr for me is 10.21.199.3:50403/0
2017-03-07 21:34:20.472123 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).connect sent my addr 10.21.199.3:0/3239701293
2017-03-07 21:34:20.472188 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).connect.authorizer_len=174 protocol=2
2017-03-07 21:34:20.472210 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).connect sending gseq=2 cseq=0 proto=24
2017-03-07 21:34:20.472242 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).connect wrote (self +) cseq, waiting for reply
2017-03-07 21:34:20.472460 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).connect got reply tag 1 connect_seq 1 global_seq 4911341 proto 24 flags 1 features 576460752032874495
2017-03-07 21:34:20.472483 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=1 pgs=0 cs=0 l=1 c=0x7f571cf1f1a0).reply.authorizer_len=36
2017-03-07 21:34:20.472527 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).connect success 1, lossy = 1, features 576460752032874495
2017-03-07 21:34:20.472551 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).connect starting reader
2017-03-07 21:34:20.472585 7f5703777700 10 client.4094747.objecter ms_handle_connect 0x7f571cf1f1a0
2017-03-07 21:34:20.472632 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:20.472650 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer encoding 1 features 576460752032874495 0x7f571cf20a70 osd_op(client.4094747.0:1 4.bc2dcdb7 rbd_data.3d31f5756d622c.0000000000005b87 [delete] snapc 0=[] ondisk+write+known_if_redirected e234774) v7
2017-03-07 21:34:20.472690 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer signed seq # 1): sig = 0
2017-03-07 21:34:20.472706 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sending 1 0x7f571cf20a70
2017-03-07 21:34:20.472744 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:20.472754 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sleeping
2017-03-07 21:34:20.472761 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader reading tag...
2017-03-07 21:34:20.675754 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ACK
2017-03-07 21:34:20.675794 7f5700670700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ack seq 1
2017-03-07 21:34:20.675804 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader reading tag...
2017-03-07 21:34:23.459117 7f5701f74700 20 -- 10.21.199.3:0/3239701293 send_keepalive con 0x7f571cf191b0, have pipe.
2017-03-07 21:34:23.459233 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:23.459271 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).write_keepalive2 14 2017-03-07 21:34:23.459270
2017-03-07 21:34:23.459330 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:23.459355 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:23.459522 7f571bdbb700  2 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got KEEPALIVE_ACK
2017-03-07 21:34:23.459572 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:25.464742 7f5703f78700 10 client.4094747.objecter tick
2017-03-07 21:34:30.464912 7f5703f78700 10 client.4094747.objecter tick
2017-03-07 21:34:33.459297 7f5701f74700 20 -- 10.21.199.3:0/3239701293 send_keepalive con 0x7f571cf191b0, have pipe.
2017-03-07 21:34:33.459409 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:33.459447 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).write_keepalive2 14 2017-03-07 21:34:33.459445
2017-03-07 21:34:33.459536 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:33.459557 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:33.459838 7f571bdbb700  2 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got KEEPALIVE_ACK
2017-03-07 21:34:33.459901 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:35.465061 7f5703f78700 10 client.4094747.objecter tick
2017-03-07 21:34:35.465102 7f5703f78700  2 client.4094747.objecter  tid 1 on osd.751 is laggy
2017-03-07 21:34:35.465114 7f5703f78700 10 client.4094747.objecter _maybe_request_map subscribing (onetime) to next osd map
2017-03-07 21:34:35.465148 7f5703f78700  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.5:6789/0 -- mon_subscribe({osdmap=234775}) v2 -- ?+0 0x7f56d4000e10 con 0x7f571cf191b0
2017-03-07 21:34:35.465189 7f5703f78700 20 -- 10.21.199.3:0/3239701293 submit_message mon_subscribe({osdmap=234775}) v2 remote, 10.21.199.5:6789/0, have pipe.
2017-03-07 21:34:35.465215 7f5703f78700  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7f56d4001410 con 0x7f571cf1f1a0
2017-03-07 21:34:35.465224 7f5703f78700 20 -- 10.21.199.3:0/3239701293 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe.
2017-03-07 21:34:35.465261 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:35.465296 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer encoding 6 features 576460752032874495 0x7f56d4000e10 mon_subscribe({osdmap=234775}) v2
2017-03-07 21:34:35.465289 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:35.465318 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer encoding 2 features 576460752032874495 0x7f56d4001410 ping magic: 0 v1
2017-03-07 21:34:35.465333 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer signed seq # 2): sig = 0
2017-03-07 21:34:35.465336 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer no session security
2017-03-07 21:34:35.465349 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sending 2 0x7f56d4001410
2017-03-07 21:34:35.465350 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sending 6 0x7f56d4000e10
2017-03-07 21:34:35.465410 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:35.465414 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:35.465421 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sleeping
2017-03-07 21:34:35.465424 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:35.665798 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got ACK
2017-03-07 21:34:35.665858 7f571bdbb700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got ack seq 6
2017-03-07 21:34:35.665869 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:35.671696 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ACK
2017-03-07 21:34:35.671739 7f5700670700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ack seq 2
2017-03-07 21:34:35.671750 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader reading tag...
2017-03-07 21:34:40.465377 7f5703f78700 10 client.4094747.objecter tick
2017-03-07 21:34:40.465417 7f5703f78700  2 client.4094747.objecter  tid 1 on osd.751 is laggy
2017-03-07 21:34:40.465428 7f5703f78700 10 client.4094747.objecter _maybe_request_map subscribing (onetime) to next osd map
2017-03-07 21:34:40.465445 7f5703f78700  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7f56d4001960 con 0x7f571cf1f1a0
2017-03-07 21:34:40.465468 7f5703f78700 20 -- 10.21.199.3:0/3239701293 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe.
2017-03-07 21:34:40.465535 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:40.465569 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer encoding 3 features 576460752032874495 0x7f56d4001960 ping magic: 0 v1
2017-03-07 21:34:40.465585 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer signed seq # 3): sig = 0
2017-03-07 21:34:40.465602 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sending 3 0x7f56d4001960
2017-03-07 21:34:40.465658 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:40.465680 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sleeping
2017-03-07 21:34:40.671749 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ACK
2017-03-07 21:34:40.671805 7f5700670700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ack seq 3
2017-03-07 21:34:40.671817 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader reading tag...
2017-03-07 21:34:43.459505 7f5701f74700 20 -- 10.21.199.3:0/3239701293 send_keepalive con 0x7f571cf191b0, have pipe.
2017-03-07 21:34:43.459616 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:43.459652 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).write_keepalive2 14 2017-03-07 21:34:43.459650
2017-03-07 21:34:43.459711 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:43.459736 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:43.460066 7f571bdbb700  2 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got KEEPALIVE_ACK
2017-03-07 21:34:43.460127 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:45.465594 7f5703f78700 10 client.4094747.objecter tick
2017-03-07 21:34:45.465633 7f5703f78700  2 client.4094747.objecter  tid 1 on osd.751 is laggy
2017-03-07 21:34:45.465657 7f5703f78700 10 client.4094747.objecter _maybe_request_map subscribing (onetime) to next osd map
2017-03-07 21:34:45.465675 7f5703f78700  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7f56d4002030 con 0x7f571cf1f1a0
2017-03-07 21:34:45.465702 7f5703f78700 20 -- 10.21.199.3:0/3239701293 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe.
2017-03-07 21:34:45.465771 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:45.465804 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer encoding 4 features 576460752032874495 0x7f56d4002030 ping magic: 0 v1
2017-03-07 21:34:45.465820 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer signed seq # 4): sig = 0
2017-03-07 21:34:45.465833 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sending 4 0x7f56d4002030
2017-03-07 21:34:45.465891 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:45.465910 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sleeping
2017-03-07 21:34:45.667774 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ACK
2017-03-07 21:34:45.667831 7f5700670700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ack seq 4
2017-03-07 21:34:45.667842 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader reading tag...
2017-03-07 21:34:50.465841 7f5703f78700 10 client.4094747.objecter tick
2017-03-07 21:34:50.465890 7f5703f78700  2 client.4094747.objecter  tid 1 on osd.751 is laggy
2017-03-07 21:34:50.465903 7f5703f78700 10 client.4094747.objecter _maybe_request_map subscribing (onetime) to next osd map
2017-03-07 21:34:50.465921 7f5703f78700  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7f56d40025f0 con 0x7f571cf1f1a0
2017-03-07 21:34:50.465948 7f5703f78700 20 -- 10.21.199.3:0/3239701293 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe.
2017-03-07 21:34:50.466013 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:50.466047 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer encoding 5 features 576460752032874495 0x7f56d40025f0 ping magic: 0 v1
2017-03-07 21:34:50.466062 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer signed seq # 5): sig = 0
2017-03-07 21:34:50.466075 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sending 5 0x7f56d40025f0
2017-03-07 21:34:50.466130 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:50.466142 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sleeping
2017-03-07 21:34:50.671796 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ACK
2017-03-07 21:34:50.671853 7f5700670700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ack seq 5
2017-03-07 21:34:50.671864 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader reading tag...
2017-03-07 21:34:53.459703 7f5701f74700 20 -- 10.21.199.3:0/3239701293 send_keepalive con 0x7f571cf191b0, have pipe.
2017-03-07 21:34:53.459814 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:53.459850 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).write_keepalive2 14 2017-03-07 21:34:53.459848
2017-03-07 21:34:53.459907 7f571bff0700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer: state = open policy.server=0
2017-03-07 21:34:53.459953 7f571bff0700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).writer sleeping
2017-03-07 21:34:53.460186 7f571bdbb700  2 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader got KEEPALIVE_ACK
2017-03-07 21:34:53.460247 7f571bdbb700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.5:6789/0 pipe(0x7f571cf17ef0 sd=4 :56520 s=2 pgs=819397 cs=1 l=1 c=0x7f571cf191b0).reader reading tag...
2017-03-07 21:34:55.466076 7f5703f78700 10 client.4094747.objecter tick
2017-03-07 21:34:55.466123 7f5703f78700  2 client.4094747.objecter  tid 1 on osd.751 is laggy
2017-03-07 21:34:55.466136 7f5703f78700 10 client.4094747.objecter _maybe_request_map subscribing (onetime) to next osd map
2017-03-07 21:34:55.466153 7f5703f78700  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7f56d4002ca0 con 0x7f571cf1f1a0
2017-03-07 21:34:55.466197 7f5703f78700 20 -- 10.21.199.3:0/3239701293 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe.
2017-03-07 21:34:55.466270 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:55.466303 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer encoding 6 features 576460752032874495 0x7f56d4002ca0 ping magic: 0 v1
2017-03-07 21:34:55.466329 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer signed seq # 6): sig = 0
2017-03-07 21:34:55.466344 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sending 6 0x7f56d4002ca0
2017-03-07 21:34:55.466402 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:34:55.466423 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sleeping
2017-03-07 21:34:55.667754 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ACK
2017-03-07 21:34:55.667810 7f5700670700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ack seq 6
2017-03-07 21:34:55.667833 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader reading tag...
2017-03-07 21:35:00.466344 7f5703f78700 10 client.4094747.objecter tick
2017-03-07 21:35:00.466389 7f5703f78700  2 client.4094747.objecter  tid 1 on osd.751 is laggy
2017-03-07 21:35:00.466401 7f5703f78700 10 client.4094747.objecter _maybe_request_map subscribing (onetime) to next osd map
2017-03-07 21:35:00.466420 7f5703f78700  1 -- 10.21.199.3:0/3239701293 --> 10.21.199.42:6820/271935 -- ping magic: 0 v1 -- ?+0 0x7f56d4003260 con 0x7f571cf1f1a0
2017-03-07 21:35:00.466444 7f5703f78700 20 -- 10.21.199.3:0/3239701293 submit_message ping magic: 0 v1 remote, 10.21.199.42:6820/271935, have pipe.
2017-03-07 21:35:00.466513 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:35:00.466547 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer encoding 7 features 576460752032874495 0x7f56d4003260 ping magic: 0 v1
2017-03-07 21:35:00.466562 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer signed seq # 7): sig = 0
2017-03-07 21:35:00.466574 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sending 7 0x7f56d4003260
2017-03-07 21:35:00.466632 7f5700771700 10 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer: state = open policy.server=0
2017-03-07 21:35:00.466654 7f5700771700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).writer sleeping
2017-03-07 21:35:00.667750 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ACK
2017-03-07 21:35:00.667807 7f5700670700 15 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader got ack seq 7
2017-03-07 21:35:00.667818 7f5700670700 20 -- 10.21.199.3:0/3239701293 >> 10.21.199.42:6820/271935 pipe(0x7f571cf1dee0 sd=5 :50403 s=2 pgs=4911341 cs=1 l=1 c=0x7f571cf1f1a0).reader reading tag...