2011-11-03 11:13:39.051596 7fb2be4fa700 -- :/0 register_entity client.? 2011-11-03 11:13:39.051660 7fb2be4fa700 -- :/0 register_entity client.? at :/0 2011-11-03 11:13:39.051763 7fb2be4fa700 -- :/0 messenger.start 2011-11-03 11:13:39.051855 7fb26d37a700 -- :/1011511 reaper_entry start 2011-11-03 11:13:39.051921 7fb26d37a700 -- :/1011511 reaper 2011-11-03 11:13:39.051940 7fb26d37a700 -- :/1011511 reaper done 2011-11-03 11:13:39.051965 7fb2be4fa700 -- :/1011511 ready :/1011511 2011-11-03 11:13:39.053199 7fb2be4fa700 -- :/1011511 connect_rank to 192.168.141.123:6789/0, creating pipe and registering 2011-11-03 11:13:39.053306 7fb2be4fa700 -- :/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=-1 pgs=0 cs=0 l=0).register_pipe 2011-11-03 11:13:39.053364 7fb26a374700 -- :/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=-1 pgs=0 cs=0 l=0).writer: state = 1 policy.server=0 2011-11-03 11:13:39.053402 7fb2be4fa700 -- :/1011511 --> 192.168.141.123:6789/0 -- auth(proto 0 30 bytes) v1 -- ?+0 0x7fb2709b8c40 con 0x7fb2709b88f0 2011-11-03 11:13:39.053444 7fb26a374700 -- :/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=-1 pgs=0 cs=0 l=0).connect 0 2011-11-03 11:13:39.053476 7fb2be4fa700 -- :/1011511 submit_message auth(proto 0 30 bytes) v1 remote 192.168.141.123:6789/0 2011-11-03 11:13:39.053512 7fb26a374700 -- :/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=0 cs=0 l=0).connecting to 192.168.141.123:6789/0 2011-11-03 11:13:39.054002 7fb26a374700 -- :/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=0 cs=0 l=0).connect read peer addr 192.168.141.123:6789/0 on socket 80 2011-11-03 11:13:39.054059 7fb26a374700 -- :/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=0 cs=0 l=0).connect peer addr for me is 192.168.141.130:50402/0 2011-11-03 11:13:39.054091 7fb26a374700 -- 192.168.141.130:0/1011511 learned my addr 192.168.141.130:0/1011511 2011-11-03 11:13:39.054136 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=0 cs=0 l=0).connect sent my addr 192.168.141.130:0/1011511 2011-11-03 11:13:39.054161 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=0 cs=0 l=0).connect sending gseq=1 cseq=0 proto=15 2011-11-03 11:13:39.054191 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=0 cs=0 l=0).connect wrote (self +) cseq, waiting for reply 2011-11-03 11:13:39.054285 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=0 cs=0 l=0).connect got reply tag 1 connect_seq 1 global_seq 375 proto 15 flags 1 2011-11-03 11:13:39.054308 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).connect success 1, lossy = 1, features 4090 2011-11-03 11:13:39.054333 7fb26a374700 -- 192.168.141.130:0/1011511 >> :/0 pipe(0x7fb270555340 sd=-1 pgs=0 cs=0 l=0).queue_received queuing pipe 2011-11-03 11:13:39.054374 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).connect starting reader 2011-11-03 11:13:39.054433 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb270555340 dequeued 0 2011-11-03 11:13:39.054473 7fb26bb77700 client.-1 ms_handle_connect on 192.168.141.123:6789/0 2011-11-03 11:13:39.054503 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.054539 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer encoding 1 0x7fb2709b8c40 auth(proto 0 30 bytes) v1 2011-11-03 11:13:39.054599 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.054678 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sending 1 0x7fb2709b8c40 2011-11-03 11:13:39.054715 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_message 0x7fb2709b8c40 2011-11-03 11:13:39.054772 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.054790 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:39.054962 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got ACK 2011-11-03 11:13:39.054989 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.055011 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got MSG 2011-11-03 11:13:39.055038 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got envelope type=18 src mon.0 front=33 data=0 off 0 2011-11-03 11:13:39.055059 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader wants 33 from dispatch throttler 0/104857600 2011-11-03 11:13:39.055082 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got front 33 2011-11-03 11:13:39.055106 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).aborted = 0 2011-11-03 11:13:39.055125 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got 33 + 0 + 0 byte message 2011-11-03 11:13:39.055208 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got message 1 0x4193ca90 auth_reply(proto 2 0 Success) v1 2011-11-03 11:13:39.055234 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:13:39.055258 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.055288 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8680 dequeued 0x4193ca90 2011-11-03 11:13:39.055327 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mon.0 192.168.141.123:6789/0 1 ==== auth_reply(proto 2 0 Success) v1 ==== 33+0+0 (2990398684 0 0) 0x4193ca90 con 0x7fb2709b88f0 2011-11-03 11:13:39.055354 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.055379 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_ack 1 2011-11-03 11:13:39.055399 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.055415 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:39.055667 7fb26bb77700 -- 192.168.141.130:0/1011511 --> 192.168.141.123:6789/0 -- auth(proto 2 32 bytes) v1 -- ?+0 0x4193ca90 con 0x7fb2709b88f0 2011-11-03 11:13:39.055690 7fb26bb77700 -- 192.168.141.130:0/1011511 submit_message auth(proto 2 32 bytes) v1 remote 192.168.141.123:6789/0 2011-11-03 11:13:39.055716 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 33 to dispatch throttler 33/104857600 2011-11-03 11:13:39.055736 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x4193ca90 2011-11-03 11:13:39.055763 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.055787 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer encoding 2 0x4193ca90 auth(proto 2 32 bytes) v1 2011-11-03 11:13:39.055825 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sending 2 0x4193ca90 2011-11-03 11:13:39.055843 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_message 0x4193ca90 2011-11-03 11:13:39.055876 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.055893 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:39.056138 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got ACK 2011-11-03 11:13:39.056161 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.056182 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got MSG 2011-11-03 11:13:39.056205 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got envelope type=18 src mon.0 front=206 data=0 off 0 2011-11-03 11:13:39.056222 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader wants 206 from dispatch throttler 0/104857600 2011-11-03 11:13:39.056245 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got front 206 2011-11-03 11:13:39.056267 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).aborted = 0 2011-11-03 11:13:39.056283 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got 206 + 0 + 0 byte message 2011-11-03 11:13:39.056310 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got message 2 0x4193ca90 auth_reply(proto 2 0 Success) v1 2011-11-03 11:13:39.056333 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:13:39.056356 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.056381 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8680 dequeued 0x4193ca90 2011-11-03 11:13:39.056411 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mon.0 192.168.141.123:6789/0 2 ==== auth_reply(proto 2 0 Success) v1 ==== 206+0+0 (3183001007 0 0) 0x4193ca90 con 0x7fb2709b88f0 2011-11-03 11:13:39.056437 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.056459 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_ack 2 2011-11-03 11:13:39.056478 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.056494 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:39.056561 7fb26bb77700 -- 192.168.141.130:0/1011511 --> 192.168.141.123:6789/0 -- auth(proto 2 165 bytes) v1 -- ?+0 0x4193ca90 con 0x7fb2709b88f0 2011-11-03 11:13:39.056581 7fb26bb77700 -- 192.168.141.130:0/1011511 submit_message auth(proto 2 165 bytes) v1 remote 192.168.141.123:6789/0 2011-11-03 11:13:39.056601 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 206 to dispatch throttler 206/104857600 2011-11-03 11:13:39.056628 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x4193ca90 2011-11-03 11:13:39.056666 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.056690 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer encoding 3 0x4193ca90 auth(proto 2 165 bytes) v1 2011-11-03 11:13:39.056714 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sending 3 0x4193ca90 2011-11-03 11:13:39.056731 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_message 0x4193ca90 2011-11-03 11:13:39.056767 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.056784 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:39.057049 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got ACK 2011-11-03 11:13:39.057072 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.057094 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got MSG 2011-11-03 11:13:39.057116 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got envelope type=18 src mon.0 front=409 data=0 off 0 2011-11-03 11:13:39.057133 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader wants 409 from dispatch throttler 0/104857600 2011-11-03 11:13:39.057156 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got front 409 2011-11-03 11:13:39.057178 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).aborted = 0 2011-11-03 11:13:39.057194 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got 409 + 0 + 0 byte message 2011-11-03 11:13:39.057220 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got message 3 0x4193cd90 auth_reply(proto 2 0 Success) v1 2011-11-03 11:13:39.057243 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:13:39.057266 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.057292 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8680 dequeued 0x4193cd90 2011-11-03 11:13:39.057320 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mon.0 192.168.141.123:6789/0 3 ==== auth_reply(proto 2 0 Success) v1 ==== 409+0+0 (4264630971 0 0) 0x4193cd90 con 0x7fb2709b88f0 2011-11-03 11:13:39.057346 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.057368 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_ack 3 2011-11-03 11:13:39.057388 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.057408 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:39.057451 7fb26bb77700 -- 192.168.141.130:0/1011511 --> 192.168.141.123:6789/0 -- mon_subscribe({mdsmap=0+,monmap=0+,osdmap=0}) v1 -- ?+0 0x7fb2709b8e20 con 0x7fb2709b88f0 2011-11-03 11:13:39.057487 7fb26bb77700 -- 192.168.141.130:0/1011511 submit_message mon_subscribe({mdsmap=0+,monmap=0+,osdmap=0}) v1 remote 192.168.141.123:6789/0 2011-11-03 11:13:39.057515 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.057548 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer encoding 4 0x7fb2709b8e20 mon_subscribe({mdsmap=0+,monmap=0+,osdmap=0}) v1 2011-11-03 11:13:39.057573 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 409 to dispatch throttler 409/104857600 2011-11-03 11:13:39.057595 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x4193cd90 2011-11-03 11:13:39.057618 7fb2be4fa700 client.5557 mounted: have osdmap 0 and mdsmap 0 2011-11-03 11:13:39.057654 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sending 4 0x7fb2709b8e20 2011-11-03 11:13:39.057673 7fb2be4fa700 client.5557 mds is 0 2011-11-03 11:13:39.057696 7fb2be4fa700 client.5557 target mds.0 not active, waiting for new mdsmap 2011-11-03 11:13:39.057722 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_message 0x7fb2709b8e20 2011-11-03 11:13:39.057765 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.057782 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:39.058116 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got ACK 2011-11-03 11:13:39.058175 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.058198 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got MSG 2011-11-03 11:13:39.058221 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got envelope type=21 src mon.0 front=535 data=0 off 0 2011-11-03 11:13:39.058237 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader wants 535 from dispatch throttler 0/104857600 2011-11-03 11:13:39.058301 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got front 535 2011-11-03 11:13:39.058325 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).aborted = 0 2011-11-03 11:13:39.058341 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got 535 + 0 + 0 byte message 2011-11-03 11:13:39.058366 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got message 4 0x4193cd90 mdsmap(e 14) v1 2011-11-03 11:13:39.058389 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:13:39.058413 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.058438 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8680 dequeued 0x4193cd90 2011-11-03 11:13:39.058466 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mon.0 192.168.141.123:6789/0 4 ==== mdsmap(e 14) v1 ==== 535+0+0 (6102218 0 0) 0x4193cd90 con 0x7fb2709b88f0 2011-11-03 11:13:39.058492 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got MSG 2011-11-03 11:13:39.058540 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.058568 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_ack 4 2011-11-03 11:13:39.058595 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got envelope type=4 src mon.0 front=187 data=0 off 0 2011-11-03 11:13:39.058619 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader wants 187 from dispatch throttler 535/104857600 2011-11-03 11:13:39.058647 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.058671 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:39.058696 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got front 187 2011-11-03 11:13:39.058724 7fb26bb77700 client.5557 handle_mds_map epoch 14 2011-11-03 11:13:39.058754 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).aborted = 0 2011-11-03 11:13:39.058774 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got 187 + 0 + 0 byte message 2011-11-03 11:13:39.058804 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got message 5 0x7fb2709b8c40 mon_map v1 2011-11-03 11:13:39.058824 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:13:39.058849 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 535 to dispatch throttler 722/104857600 2011-11-03 11:13:39.058873 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x4193cd90 2011-11-03 11:13:39.058892 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8680 dequeued 0x7fb2709b8c40 2011-11-03 11:13:39.058914 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mon.0 192.168.141.123:6789/0 5 ==== mon_map v1 ==== 187+0+0 (2564923774 0 0) 0x7fb2709b8c40 con 0x7fb2709b88f0 2011-11-03 11:13:39.058939 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.058967 7fb2be4fa700 client.5557 mds is 0 2011-11-03 11:13:39.058993 7fb2be4fa700 client.5557 opening session to mds.0 2011-11-03 11:13:39.059023 7fb2be4fa700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_open) v1 -- ?+0 0x7fb2709b7390 2011-11-03 11:13:39.059051 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got MSG 2011-11-03 11:13:39.059081 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 187 to dispatch throttler 187/104857600 2011-11-03 11:13:39.059103 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x7fb2709b8c40 2011-11-03 11:13:39.059131 7fb2be4fa700 -- 192.168.141.130:0/1011511 submit_message client_session(request_open) v1 remote, 192.168.141.123:6802/19591, new pipe. 2011-11-03 11:13:39.059155 7fb2be4fa700 -- 192.168.141.130:0/1011511 connect_rank to 192.168.141.123:6802/19591, creating pipe and registering 2011-11-03 11:13:39.059182 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.059212 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_ack 5 2011-11-03 11:13:39.059236 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.059270 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:39.059313 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=-1 pgs=0 cs=0 l=0).writer: state = 1 policy.server=0 2011-11-03 11:13:39.059350 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=-1 pgs=0 cs=0 l=0).connect 0 2011-11-03 11:13:39.059381 7fb2be4fa700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=-1 pgs=0 cs=0 l=0).register_pipe 2011-11-03 11:13:39.059420 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got envelope type=41 src mon.0 front=6122 data=0 off 0 2011-11-03 11:13:39.059450 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader wants 6122 from dispatch throttler 0/104857600 2011-11-03 11:13:39.059474 7fb2be4fa700 client.5557 waiting for session to mds.0 to open 2011-11-03 11:13:39.059521 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got front 6122 2011-11-03 11:13:39.059555 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=0 cs=0 l=0).connecting to 192.168.141.123:6802/19591 2011-11-03 11:13:39.059588 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).aborted = 0 2011-11-03 11:13:39.059612 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got 6122 + 0 + 0 byte message 2011-11-03 11:13:39.059679 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got message 6 0x7fb2709b9d70 osd_map(16..16 src has 1..16) v2 2011-11-03 11:13:39.059703 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:13:39.059734 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.059762 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8680 dequeued 0x7fb2709b9d70 2011-11-03 11:13:39.059795 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mon.0 192.168.141.123:6789/0 6 ==== osd_map(16..16 src has 1..16) v2 ==== 6122+0+0 (849204460 0 0) 0x7fb2709b9d70 con 0x7fb2709b88f0 2011-11-03 11:13:39.059820 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got MSG 2011-11-03 11:13:39.059852 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got envelope type=16 src mon.0 front=20 data=0 off 0 2011-11-03 11:13:39.059868 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader wants 20 from dispatch throttler 6122/104857600 2011-11-03 11:13:39.059891 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got front 20 2011-11-03 11:13:39.059915 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).aborted = 0 2011-11-03 11:13:39.059931 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got 20 + 0 + 0 byte message 2011-11-03 11:13:39.059955 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.059975 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_ack 6 2011-11-03 11:13:39.060009 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.060026 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:39.060048 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader got message 7 0x7fb2709ba0c0 mon_subscribe_ack(300s) v1 2011-11-03 11:13:39.060067 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:13:39.060089 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader reading tag... 2011-11-03 11:13:39.060119 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 6122 to dispatch throttler 6142/104857600 2011-11-03 11:13:39.060144 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x7fb2709b9d70 2011-11-03 11:13:39.060170 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.060197 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_ack 7 2011-11-03 11:13:39.060221 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8680 dequeued 0x7fb2709ba0c0 2011-11-03 11:13:39.060258 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=0 cs=0 l=0).connect read peer addr 192.168.141.123:6802/19591 on socket 81 2011-11-03 11:13:39.060286 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=0 cs=0 l=0).connect peer addr for me is 192.168.141.130:46025/0 2011-11-03 11:13:39.060316 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mon.0 192.168.141.123:6789/0 7 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3759263596 0 0) 0x7fb2709ba0c0 con 0x7fb2709b88f0 2011-11-03 11:13:39.060354 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=0 cs=0 l=0).connect sent my addr 192.168.141.130:0/1011511 2011-11-03 11:13:39.060401 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:39.060427 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:39.060454 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=0 cs=0 l=0).connect.authorizer_len=174 protocol=2 2011-11-03 11:13:39.060483 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=0 cs=0 l=0).connect sending gseq=2 cseq=0 proto=32 2011-11-03 11:13:39.060508 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 20 to dispatch throttler 20/104857600 2011-11-03 11:13:39.060534 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x7fb2709ba0c0 2011-11-03 11:13:39.060561 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=0 cs=0 l=0).connect wrote (self +) cseq, waiting for reply 2011-11-03 11:13:39.060835 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=0 cs=0 l=0).connect got reply tag 1 connect_seq 1 global_seq 1174 proto 32 flags 0 2011-11-03 11:13:39.060857 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=0 cs=0 l=0).reply.authorizer_len=36 2011-11-03 11:13:39.060953 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).connect success 1, lossy = 0, features 4090 2011-11-03 11:13:39.060973 7fb26a172700 -- 192.168.141.130:0/1011511 >> :/0 pipe(0x7fb270555340 sd=-1 pgs=0 cs=0 l=0).queue_received queuing pipe 2011-11-03 11:13:39.061010 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).connect starting reader 2011-11-03 11:13:39.061038 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb270555340 dequeued 0 2011-11-03 11:13:39.061064 7fb26bb77700 client.5557 ms_handle_connect on 192.168.141.123:6802/19591 2011-11-03 11:13:39.061085 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.061114 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 1 0x7fb2709b7390 client_session(request_open) v1 2011-11-03 11:13:39.061139 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 1 0x7fb2709b7390 2011-11-03 11:13:39.061162 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x7fb2709b7390 2011-11-03 11:13:39.061197 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:13:39.061233 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.061255 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:13:39.258113 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:13:39.258144 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:13:39.258167 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:13:39.258191 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:13:39.258208 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:13:39.258231 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 28 2011-11-03 11:13:39.258305 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:13:39.258324 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:13:39.258350 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 1 0x417e4ad0 client_session(open) v1 2011-11-03 11:13:39.258389 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:13:39.258415 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:13:39.258443 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x417e4ad0 2011-11-03 11:13:39.258473 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 1 ==== client_session(open) v1 ==== 28+0+0 (2719320884 0 0) 0x417e4ad0 con 0x7fb2709b9a40 2011-11-03 11:13:39.258500 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.258542 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 1 2011-11-03 11:13:39.258567 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.258587 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:13:39.258610 7fb26bb77700 client.5557 handle_client_session client_session(open) v1 2011-11-03 11:13:39.258648 7fb26bb77700 client.5557 renew_caps mds.0 2011-11-03 11:13:39.258673 7fb26bb77700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_renewcaps seq 1) v1 -- ?+0 0x41deb720 2011-11-03 11:13:39.258696 7fb26bb77700 -- 192.168.141.130:0/1011511 submit_message client_session(request_renewcaps seq 1) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:13:39.258717 7fb26bb77700 client.5557 connect_mds_targets for mds.0 2011-11-03 11:13:39.258744 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.258773 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 2 0x41deb720 client_session(request_renewcaps seq 1) v1 2011-11-03 11:13:39.258798 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 2 0x41deb720 2011-11-03 11:13:39.258820 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x41deb720 2011-11-03 11:13:39.258850 7fb2be4fa700 client.5557 send_request rebuilding request 1 for mds.0 2011-11-03 11:13:39.258884 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:13:39.258907 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x417e4ad0 2011-11-03 11:13:39.258935 7fb2be4fa700 client.5557 encode_cap_releases enter (req: 0x7fb2709b9750, mds: 0) 2011-11-03 11:13:39.258969 7fb2be4fa700 client.5557 send_request set sent_stamp to 2011-11-03 11:13:39.258956 2011-11-03 11:13:39.259010 7fb2be4fa700 client.5557 send_request client_request(unknown.0:1 getattr pAsLsXsFs #1) v1 to mds.0 2011-11-03 11:13:39.259036 7fb2be4fa700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5557:1 getattr pAsLsXsFs #1) v1 -- ?+0 0x7fb2709b7390 2011-11-03 11:13:39.259064 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:13:39.259101 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.259124 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:13:39.259159 7fb2be4fa700 -- 192.168.141.130:0/1011511 submit_message client_request(client.5557:1 getattr pAsLsXsFs #1) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:13:39.259186 7fb2be4fa700 client.5557 awaiting reply|forward|kick on 0x7fb2be4f9090 2011-11-03 11:13:39.259214 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.259254 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 3 0x7fb2709b7390 client_request(client.5557:1 getattr pAsLsXsFs #1) v1 2011-11-03 11:13:39.259280 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:13:39.259318 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:13:39.259362 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 3 0x7fb2709b7390 2011-11-03 11:13:39.259392 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x7fb2709b7390 2011-11-03 11:13:39.259419 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:13:39.259450 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:13:39.259479 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.259514 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:13:39.259539 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 28 2011-11-03 11:13:39.259567 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:13:39.259584 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:13:39.259607 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 2 0x417e4ad0 client_session(renewcaps seq 1) v1 2011-11-03 11:13:39.259644 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:13:39.259669 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:13:39.259695 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x417e4ad0 2011-11-03 11:13:39.259722 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (2614397064 0 0) 0x417e4ad0 con 0x7fb2709b9a40 2011-11-03 11:13:39.259740 7fb26bb77700 client.5557 handle_client_session client_session(renewcaps seq 1) v1 2011-11-03 11:13:39.259766 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.259792 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 2 2011-11-03 11:13:39.259816 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.259836 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:13:39.259861 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:13:39.259883 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x417e4ad0 2011-11-03 11:13:39.259907 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:13:39.259942 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:13:39.259965 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:13:39.260002 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=26 src mds.0 front=316 data=0 off 0 2011-11-03 11:13:39.260019 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 316 from dispatch throttler 0/104857600 2011-11-03 11:13:39.260042 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 316 2011-11-03 11:13:39.260064 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:13:39.260081 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 316 + 0 + 0 byte message 2011-11-03 11:13:39.260118 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 3 0x41c33d20 client_reply(???:1 = 0 Success) v1 2011-11-03 11:13:39.260141 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:13:39.260166 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:13:39.260191 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x41c33d20 2011-11-03 11:13:39.260221 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 3 ==== client_reply(???:1 = 0 Success) v1 ==== 316+0+0 (1666487100 0 0) 0x41c33d20 con 0x7fb2709b9a40 2011-11-03 11:13:39.260247 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.260273 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 3 2011-11-03 11:13:39.260295 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:39.260315 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:13:39.260338 7fb26bb77700 client.5557 handle_client_reply got a reply. Safe:1 tid 1 2011-11-03 11:13:39.260367 7fb26bb77700 client.5557 insert_trace from 2011-11-03 11:13:39.258956 mds.0 is_target=1 is_dentry=0 2011-11-03 11:13:39.260380 7fb26bb77700 client.5557 features 0xffa 2011-11-03 11:13:39.260397 7fb26bb77700 client.5557 update_snap_trace len 48 2011-11-03 11:13:39.260429 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=2 c=0 seq=0 parent=0 my_snaps=[] cached_snapc=0=[]) seq 1 > 0 2011-11-03 11:13:39.260444 7fb26bb77700 client.5557 flushing caps on snaprealm(1 nref=2 c=0 seq=0 parent=0 my_snaps=[] cached_snapc=0=[]) 2011-11-03 11:13:39.260466 7fb26bb77700 client.5557 invalidate_snaprealm_and_children snaprealm(1 nref=2 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=0=[]) 2011-11-03 11:13:39.260481 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=2 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=0=[]) self|parent updated 2011-11-03 11:13:39.260498 7fb26bb77700 client.5557 snapc 1=[] 2011-11-03 11:13:39.260513 7fb26bb77700 client.5557 hrm is_target=1 is_dentry=0 2011-11-03 11:13:39.260545 7fb26bb77700 inode.get on 0x41deb720 1.head now 1 2011-11-03 11:13:39.260575 7fb26bb77700 client.5557 add_update_inode adding 1.head(ref=1 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41deb720) caps pAsLsXsFs 2011-11-03 11:13:39.260590 7fb26bb77700 client.5557 dir hash is 2 2011-11-03 11:13:39.260614 7fb26bb77700 client.5557 update_inode_file_bits 1.head(ref=1 cap_refs={} open={} mode=40755 size=0 mtime=0.000000 caps=- 0x41deb720) - mtime 2011-11-01 21:12:55.797267 2011-11-03 11:13:39.260649 7fb26bb77700 client.5557 add_update_cap first one, opened snaprealm 0x41c33f90 2011-11-03 11:13:39.260689 7fb26bb77700 client.5557 add_update_cap issued - -> pAsLsXsFs from mds.0 on 1.head(ref=1 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) 2011-11-03 11:13:39.260707 7fb26bb77700 client.5557 handle_client_reply signalling caller 0x7fb2be4f9090 2011-11-03 11:13:39.260725 7fb26bb77700 client.5557 handle_client_reply awaiting kickback on tid 1 0x7fb26bb76490 2011-11-03 11:13:39.260760 7fb2be4fa700 client.5557 sendrecv kickback on tid 1 0x7fb26bb76490 2011-11-03 11:13:39.260780 7fb2be4fa700 client.5557 lat 0.001821 2011-11-03 11:13:39.260803 7fb2be4fa700 client.5557 root getattr result=0 2011-11-03 11:13:39.260820 7fb2be4fa700 inode.get on 0x41deb720 1.head now 2 2011-11-03 11:13:39.260833 7fb2be4fa700 client.5557 _ll_get 0x41deb720 1 -> 1 2011-11-03 11:13:39.260876 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 316 to dispatch throttler 316/104857600 2011-11-03 11:13:39.260895 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x41c33d20 2011-11-03 11:13:39.261161 7fb2be4fa700 client.5557 path_walk #1 2011-11-03 11:13:39.261180 7fb2be4fa700 client.5557 chdir(/) cwd now 1 2011-11-03 11:13:40.057691 7fb26cb79700 client.5557 renew_caps() 2011-11-03 11:13:40.057758 7fb26cb79700 client.5557 renew_caps requesting from mds.0 2011-11-03 11:13:40.057772 7fb26cb79700 client.5557 renew_caps mds.0 2011-11-03 11:13:40.057814 7fb26cb79700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_renewcaps seq 2) v1 -- ?+0 0x41c33bd0 2011-11-03 11:13:40.057836 7fb26cb79700 -- 192.168.141.130:0/1011511 submit_message client_session(request_renewcaps seq 2) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:13:40.057899 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:40.057966 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 4 0x41c33bd0 client_session(request_renewcaps seq 2) v1 2011-11-03 11:13:40.057998 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 4 0x41c33bd0 2011-11-03 11:13:40.058019 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x41c33bd0 2011-11-03 11:13:40.058062 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:40.058079 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:13:40.058215 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:13:40.058259 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:13:40.058286 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:13:40.058311 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:13:40.058329 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:13:40.058353 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 28 2011-11-03 11:13:40.058376 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:13:40.058427 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:13:40.058459 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 4 0x41c33bd0 client_session(renewcaps seq 2) v1 2011-11-03 11:13:40.058486 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:13:40.058512 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:13:40.058570 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:40.058629 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 4 2011-11-03 11:13:40.058659 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x41c33bd0 2011-11-03 11:13:40.058689 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 4 ==== client_session(renewcaps seq 2) v1 ==== 28+0+0 (1357429227 0 0) 0x41c33bd0 con 0x7fb2709b9a40 2011-11-03 11:13:40.058711 7fb26bb77700 client.5557 handle_client_session client_session(renewcaps seq 2) v1 2011-11-03 11:13:40.058738 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:13:40.058758 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x41c33bd0 2011-11-03 11:13:40.058790 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:13:40.058813 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:13:42.053239 7fb26b376700 -- 192.168.141.130:0/1011511 send_keepalive con 0x7fb2709b88f0, have pipe. 2011-11-03 11:13:42.053348 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:42.053418 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_keepalive 2011-11-03 11:13:42.053450 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:42.053469 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:13:52.053425 7fb26b376700 -- 192.168.141.130:0/1011511 send_keepalive con 0x7fb2709b88f0, have pipe. 2011-11-03 11:13:52.053525 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:52.053589 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_keepalive 2011-11-03 11:13:52.053621 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:13:52.053644 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:14:00.059115 7fb26cb79700 client.5557 renew_caps() 2011-11-03 11:14:00.059199 7fb26cb79700 client.5557 renew_caps requesting from mds.0 2011-11-03 11:14:00.059216 7fb26cb79700 client.5557 renew_caps mds.0 2011-11-03 11:14:00.059250 7fb26cb79700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_renewcaps seq 3) v1 -- ?+0 0x41c33bd0 2011-11-03 11:14:00.059287 7fb26cb79700 -- 192.168.141.130:0/1011511 submit_message client_session(request_renewcaps seq 3) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:00.059368 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:00.059434 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 5 0x41c33bd0 client_session(request_renewcaps seq 3) v1 2011-11-03 11:14:00.059467 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 5 0x41c33bd0 2011-11-03 11:14:00.059488 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x41c33bd0 2011-11-03 11:14:00.059540 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:00.059557 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:00.059739 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:00.059829 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:00.059854 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:00.059881 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:14:00.059899 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:14:00.059926 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 28 2011-11-03 11:14:00.059950 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:00.059967 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:14:00.060016 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 5 0x41c33bd0 client_session(renewcaps seq 3) v1 2011-11-03 11:14:00.060043 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:00.060071 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:00.060103 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x41c33bd0 2011-11-03 11:14:00.060134 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 5 ==== client_session(renewcaps seq 3) v1 ==== 28+0+0 (3936844645 0 0) 0x41c33bd0 con 0x7fb2709b9a40 2011-11-03 11:14:00.060157 7fb26bb77700 client.5557 handle_client_session client_session(renewcaps seq 3) v1 2011-11-03 11:14:00.060189 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:00.060217 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 5 2011-11-03 11:14:00.060245 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:00.060290 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:00.060319 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:14:00.060340 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x41c33bd0 2011-11-03 11:14:02.053595 7fb26b376700 -- 192.168.141.130:0/1011511 send_keepalive con 0x7fb2709b88f0, have pipe. 2011-11-03 11:14:02.053680 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:02.053738 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_keepalive 2011-11-03 11:14:02.053771 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:02.053789 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.475156 7fb269f70700 client.5557 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:06.475257 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:06.475297 7fb269f70700 client.5557 0 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:06.475335 7fb269f70700 client.5557 _lookup on #1/tmp 2011-11-03 11:14:06.475364 7fb269f70700 client.5557 choose_target_mds starting with req->inode 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) 2011-11-03 11:14:06.475395 7fb269f70700 client.5557 choose_target_mds inode dir hash is 2 on tmp => 3207142759 2011-11-03 11:14:06.475420 7fb269f70700 client.5557 choose_target_mds 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) is_hash=1 hash=3207142759 2011-11-03 11:14:06.475444 7fb269f70700 client.5557 choose_target_mds from caps on inode 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) 2011-11-03 11:14:06.475458 7fb269f70700 client.5557 mds is 0 2011-11-03 11:14:06.475474 7fb269f70700 client.5557 send_request rebuilding request 2 for mds.0 2011-11-03 11:14:06.475491 7fb269f70700 client.5557 encode_cap_releases enter (req: 0x41c350b0, mds: 0) 2011-11-03 11:14:06.475509 7fb269f70700 client.5557 send_request set sent_stamp to 2011-11-03 11:14:06.475504 2011-11-03 11:14:06.475527 7fb269f70700 client.5557 send_request client_request(unknown.0:2 lookup #1/tmp) v1 to mds.0 2011-11-03 11:14:06.475553 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5557:2 lookup #1/tmp) v1 -- ?+0 0x41c353d0 2011-11-03 11:14:06.475577 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_request(client.5557:2 lookup #1/tmp) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.475605 7fb269f70700 client.5557 awaiting reply|forward|kick on 0x7fb269f6f030 2011-11-03 11:14:06.475650 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.475727 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 6 0x41c353d0 client_request(client.5557:2 lookup #1/tmp) v1 2011-11-03 11:14:06.475763 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 6 0x41c353d0 2011-11-03 11:14:06.475785 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x41c353d0 2011-11-03 11:14:06.475878 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.475895 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.476306 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.476341 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.476364 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.476389 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=26 src mds.0 front=582 data=0 off 0 2011-11-03 11:14:06.476411 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 582 from dispatch throttler 0/104857600 2011-11-03 11:14:06.476438 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 582 2011-11-03 11:14:06.476462 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.476479 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 582 + 0 + 0 byte message 2011-11-03 11:14:06.476526 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 6 0x41c35690 client_reply(???:2 = 0 Success) v1 2011-11-03 11:14:06.476558 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.476588 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.476635 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x41c35690 2011-11-03 11:14:06.476705 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 6 ==== client_reply(???:2 = 0 Success) v1 ==== 582+0+0 (895897136 0 0) 0x41c35690 con 0x7fb2709b9a40 2011-11-03 11:14:06.476728 7fb26bb77700 client.5557 handle_client_reply got a reply. Safe:1 tid 2 2011-11-03 11:14:06.476750 7fb26bb77700 client.5557 insert_trace from 2011-11-03 11:14:06.475504 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.476768 7fb26bb77700 client.5557 features 0xffa 2011-11-03 11:14:06.476786 7fb26bb77700 client.5557 update_snap_trace len 48 2011-11-03 11:14:06.476813 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.476843 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 6 2011-11-03 11:14:06.476868 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.476889 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.476915 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=3 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.476933 7fb26bb77700 client.5557 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.476980 7fb26bb77700 client.5557 add_update_inode adding 1000000005c.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41c359c0) caps pAsLsXsFs 2011-11-03 11:14:06.477017 7fb26bb77700 client.5557 dir hash is 2 2011-11-03 11:14:06.477036 7fb26bb77700 client.5557 update_inode_file_bits 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x41c359c0) - mtime 2011-11-01 21:06:06.505221 2011-11-03 11:14:06.477052 7fb26bb77700 client.5557 add_update_cap first one, opened snaprealm 0x41c33f90 2011-11-03 11:14:06.477085 7fb26bb77700 client.5557 add_update_cap issued - -> pAsLsXsFs from mds.0 on 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41c359c0) 2011-11-03 11:14:06.477111 7fb26bb77700 client.5557 add_update_inode had 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) caps pAsLsXsFs 2011-11-03 11:14:06.477126 7fb26bb77700 client.5557 dir hash is 2 2011-11-03 11:14:06.477150 7fb26bb77700 client.5557 update_inode_file_bits 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) pAsLsXsFs mtime 2011-11-01 21:12:55.797267 2011-11-03 11:14:06.477178 7fb26bb77700 client.5557 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) 2011-11-03 11:14:06.477208 7fb26bb77700 client.5557 got dirfrag map for 1 frag * to mds 0 2011-11-03 11:14:06.477253 7fb26bb77700 client.5557 insert_dentry_inode 'tmp' vino 1000000005c.head in dir 1.head dn 0 2011-11-03 11:14:06.477268 7fb26bb77700 inode.get on 0x41c359c0 1000000005c.head now 1 2011-11-03 11:14:06.477309 7fb26bb77700 client.5557 link dir 0x41deb720 'tmp' to inode 0x41c359c0 dn 0x41c35590 (new dn) 2011-11-03 11:14:06.477322 7fb26bb77700 inode.get on 0x41c359c0 1000000005c.head now 2 2011-11-03 11:14:06.477338 7fb26bb77700 client.5557 link inode 0x41c359c0 parents now 0x41c35590 2011-11-03 11:14:06.477352 7fb26bb77700 inode.put on 0x41c359c0 1000000005c.head now 1 2011-11-03 11:14:06.477363 7fb26bb77700 client.5557 setting dn offset to 2 2011-11-03 11:14:06.477384 7fb26bb77700 client.5557 handle_client_reply signalling caller 0x7fb269f6f030 2011-11-03 11:14:06.477404 7fb26bb77700 client.5557 handle_client_reply awaiting kickback on tid 2 0x7fb26bb76490 2011-11-03 11:14:06.477429 7fb269f70700 client.5557 sendrecv kickback on tid 2 0x7fb26bb76490 2011-11-03 11:14:06.477450 7fb269f70700 client.5557 lat 0.001942 2011-11-03 11:14:06.477469 7fb269f70700 client.5557 _lookup res is 0 2011-11-03 11:14:06.477504 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.477527 7fb269f70700 client.5557 1 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:06.477545 7fb269f70700 client.5557 _lookup on #1000000005c/hadoop-nwatkins 2011-11-03 11:14:06.477568 7fb269f70700 client.5557 choose_target_mds starting with req->inode 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.477581 7fb269f70700 client.5557 choose_target_mds inode dir hash is 2 on hadoop-nwatkins => 2416329197 2011-11-03 11:14:06.477603 7fb269f70700 client.5557 choose_target_mds 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) is_hash=1 hash=2416329197 2011-11-03 11:14:06.477626 7fb269f70700 client.5557 choose_target_mds from caps on inode 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.477661 7fb269f70700 client.5557 mds is 0 2011-11-03 11:14:06.477674 7fb269f70700 client.5557 send_request rebuilding request 3 for mds.0 2011-11-03 11:14:06.477688 7fb269f70700 client.5557 encode_cap_releases enter (req: 0x417e5480, mds: 0) 2011-11-03 11:14:06.477703 7fb269f70700 client.5557 send_request set sent_stamp to 2011-11-03 11:14:06.477699 2011-11-03 11:14:06.477717 7fb269f70700 client.5557 send_request client_request(unknown.0:3 lookup #1000000005c/hadoop-nwatkins) v1 to mds.0 2011-11-03 11:14:06.477736 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5557:3 lookup #1000000005c/hadoop-nwatkins) v1 -- ?+0 0x417e57a0 2011-11-03 11:14:06.477755 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_request(client.5557:3 lookup #1000000005c/hadoop-nwatkins) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.477774 7fb269f70700 client.5557 awaiting reply|forward|kick on 0x7fb269f6f030 2011-11-03 11:14:06.477802 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.477832 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 7 0x417e57a0 client_request(client.5557:3 lookup #1000000005c/hadoop-nwatkins) v1 2011-11-03 11:14:06.477862 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 582 to dispatch throttler 582/104857600 2011-11-03 11:14:06.477886 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x41c35690 2011-11-03 11:14:06.477912 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 7 0x417e57a0 2011-11-03 11:14:06.477935 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x417e57a0 2011-11-03 11:14:06.477963 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.477979 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.478295 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.478321 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.478343 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.478366 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=26 src mds.0 front=594 data=0 off 0 2011-11-03 11:14:06.478383 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 594 from dispatch throttler 0/104857600 2011-11-03 11:14:06.478407 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 594 2011-11-03 11:14:06.478434 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.478450 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 594 + 0 + 0 byte message 2011-11-03 11:14:06.478477 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 7 0x41c35690 client_reply(???:3 = 0 Success) v1 2011-11-03 11:14:06.478501 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.478541 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.478571 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.478598 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 7 2011-11-03 11:14:06.478626 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x41c35690 2011-11-03 11:14:06.478657 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 7 ==== client_reply(???:3 = 0 Success) v1 ==== 594+0+0 (4171668641 0 0) 0x41c35690 con 0x7fb2709b9a40 2011-11-03 11:14:06.478677 7fb26bb77700 client.5557 handle_client_reply got a reply. Safe:1 tid 3 2011-11-03 11:14:06.478702 7fb26bb77700 client.5557 insert_trace from 2011-11-03 11:14:06.477699 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.478721 7fb26bb77700 client.5557 features 0xffa 2011-11-03 11:14:06.478739 7fb26bb77700 client.5557 update_snap_trace len 48 2011-11-03 11:14:06.478764 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=4 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.478782 7fb26bb77700 client.5557 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.478809 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.478834 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.478872 7fb26bb77700 client.5557 add_update_inode adding 1000000005d.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41c350b0) caps pAsLsXsFs 2011-11-03 11:14:06.478892 7fb26bb77700 client.5557 dir hash is 2 2011-11-03 11:14:06.478910 7fb26bb77700 client.5557 update_inode_file_bits 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x41c350b0) - mtime 2011-11-01 21:06:06.505963 2011-11-03 11:14:06.478926 7fb26bb77700 client.5557 add_update_cap first one, opened snaprealm 0x41c33f90 2011-11-03 11:14:06.478957 7fb26bb77700 client.5557 add_update_cap issued - -> pAsLsXsFs from mds.0 on 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41c350b0) 2011-11-03 11:14:06.478983 7fb26bb77700 client.5557 add_update_inode had 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) caps pAsLsXsFs 2011-11-03 11:14:06.479016 7fb26bb77700 client.5557 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.479030 7fb26bb77700 client.5557 got dirfrag map for 1000000005c frag * to mds -1 2011-11-03 11:14:06.479050 7fb26bb77700 client.5557 insert_dentry_inode 'hadoop-nwatkins' vino 1000000005d.head in dir 1000000005c.head dn 0 2011-11-03 11:14:06.479064 7fb26bb77700 inode.get on 0x41c350b0 1000000005d.head now 1 2011-11-03 11:14:06.479083 7fb26bb77700 client.5557 link dir 0x41c359c0 'hadoop-nwatkins' to inode 0x41c350b0 dn 0x417e6170 (new dn) 2011-11-03 11:14:06.479096 7fb26bb77700 inode.get on 0x41c350b0 1000000005d.head now 2 2011-11-03 11:14:06.479109 7fb26bb77700 client.5557 link inode 0x41c350b0 parents now 0x417e6170 2011-11-03 11:14:06.479123 7fb26bb77700 inode.put on 0x41c350b0 1000000005d.head now 1 2011-11-03 11:14:06.479138 7fb26bb77700 client.5557 setting dn offset to 2 2011-11-03 11:14:06.479154 7fb26bb77700 client.5557 handle_client_reply signalling caller 0x7fb269f6f030 2011-11-03 11:14:06.479186 7fb26bb77700 client.5557 handle_client_reply awaiting kickback on tid 3 0x7fb26bb76490 2011-11-03 11:14:06.479215 7fb269f70700 client.5557 sendrecv kickback on tid 3 0x7fb26bb76490 2011-11-03 11:14:06.479236 7fb269f70700 client.5557 lat 0.001534 2011-11-03 11:14:06.479252 7fb269f70700 client.5557 _lookup res is 0 2011-11-03 11:14:06.479285 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.479307 7fb269f70700 client.5557 2 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:06.479325 7fb269f70700 client.5557 _lookup on #1000000005d/mapred 2011-11-03 11:14:06.479348 7fb269f70700 client.5557 choose_target_mds starting with req->inode 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.479362 7fb269f70700 client.5557 choose_target_mds inode dir hash is 2 on mapred => 1878159916 2011-11-03 11:14:06.479383 7fb269f70700 client.5557 choose_target_mds 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) is_hash=1 hash=1878159916 2011-11-03 11:14:06.479406 7fb269f70700 client.5557 choose_target_mds from caps on inode 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.479418 7fb269f70700 client.5557 mds is 0 2011-11-03 11:14:06.479431 7fb269f70700 client.5557 send_request rebuilding request 4 for mds.0 2011-11-03 11:14:06.479445 7fb269f70700 client.5557 encode_cap_releases enter (req: 0x417e5770, mds: 0) 2011-11-03 11:14:06.479460 7fb269f70700 client.5557 send_request set sent_stamp to 2011-11-03 11:14:06.479456 2011-11-03 11:14:06.479474 7fb269f70700 client.5557 send_request client_request(unknown.0:4 lookup #1000000005d/mapred) v1 to mds.0 2011-11-03 11:14:06.479492 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5557:4 lookup #1000000005d/mapred) v1 -- ?+0 0x41c35690 2011-11-03 11:14:06.479511 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_request(client.5557:4 lookup #1000000005d/mapred) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.479532 7fb269f70700 client.5557 awaiting reply|forward|kick on 0x7fb269f6f030 2011-11-03 11:14:06.479560 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.479593 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 8 0x41c35690 client_request(client.5557:4 lookup #1000000005d/mapred) v1 2011-11-03 11:14:06.479618 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 594 to dispatch throttler 594/104857600 2011-11-03 11:14:06.479641 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x41c35690 2011-11-03 11:14:06.479668 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 8 0x41c35690 2011-11-03 11:14:06.479692 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x41c35690 2011-11-03 11:14:06.479728 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.479744 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.480016 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.480048 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.480070 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.480093 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=26 src mds.0 front=585 data=0 off 0 2011-11-03 11:14:06.480110 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 585 from dispatch throttler 0/104857600 2011-11-03 11:14:06.480137 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 585 2011-11-03 11:14:06.480160 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.480177 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 585 + 0 + 0 byte message 2011-11-03 11:14:06.480205 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 8 0x417e6250 client_reply(???:4 = 0 Success) v1 2011-11-03 11:14:06.480228 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.480251 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.480280 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.480304 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 8 2011-11-03 11:14:06.480328 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x417e6250 2011-11-03 11:14:06.480358 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 8 ==== client_reply(???:4 = 0 Success) v1 ==== 585+0+0 (1905764332 0 0) 0x417e6250 con 0x7fb2709b9a40 2011-11-03 11:14:06.480377 7fb26bb77700 client.5557 handle_client_reply got a reply. Safe:1 tid 4 2011-11-03 11:14:06.480395 7fb26bb77700 client.5557 insert_trace from 2011-11-03 11:14:06.479456 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.480411 7fb26bb77700 client.5557 features 0xffa 2011-11-03 11:14:06.480432 7fb26bb77700 client.5557 update_snap_trace len 48 2011-11-03 11:14:06.480451 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=5 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.480468 7fb26bb77700 client.5557 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.480495 7fb26bb77700 client.5557 add_update_inode adding 1000000005e.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x417e6510) caps pAsLsXs 2011-11-03 11:14:06.480512 7fb26bb77700 client.5557 dir hash is 2 2011-11-03 11:14:06.480533 7fb26bb77700 client.5557 update_inode_file_bits 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x417e6510) - mtime 2011-11-03 11:13:37.993864 2011-11-03 11:14:06.480552 7fb26bb77700 client.5557 add_update_cap first one, opened snaprealm 0x41c33f90 2011-11-03 11:14:06.480587 7fb26bb77700 client.5557 add_update_cap issued - -> pAsLsXs from mds.0 on 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) 0x417e6510) 2011-11-03 11:14:06.480631 7fb26bb77700 client.5557 add_update_inode had 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) caps pAsLsXsFs 2011-11-03 11:14:06.480667 7fb26bb77700 client.5557 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.480684 7fb26bb77700 client.5557 got dirfrag map for 1000000005d frag * to mds -1 2011-11-03 11:14:06.480705 7fb26bb77700 client.5557 insert_dentry_inode 'mapred' vino 1000000005e.head in dir 1000000005d.head dn 0 2011-11-03 11:14:06.480726 7fb26bb77700 inode.get on 0x417e6510 1000000005e.head now 1 2011-11-03 11:14:06.480745 7fb26bb77700 client.5557 link dir 0x41c350b0 'mapred' to inode 0x417e6510 dn 0x417e56e0 (new dn) 2011-11-03 11:14:06.480761 7fb26bb77700 inode.get on 0x417e6510 1000000005e.head now 2 2011-11-03 11:14:06.480779 7fb26bb77700 client.5557 link inode 0x417e6510 parents now 0x417e56e0 2011-11-03 11:14:06.480795 7fb26bb77700 inode.put on 0x417e6510 1000000005e.head now 1 2011-11-03 11:14:06.480812 7fb26bb77700 client.5557 setting dn offset to 2 2011-11-03 11:14:06.480829 7fb26bb77700 client.5557 handle_client_reply signalling caller 0x7fb269f6f030 2011-11-03 11:14:06.480858 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.480882 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.480905 7fb26bb77700 client.5557 handle_client_reply awaiting kickback on tid 4 0x7fb26bb76490 2011-11-03 11:14:06.480938 7fb269f70700 client.5557 sendrecv kickback on tid 4 0x7fb26bb76490 2011-11-03 11:14:06.480958 7fb269f70700 client.5557 lat 0.001499 2011-11-03 11:14:06.480973 7fb269f70700 client.5557 _lookup res is 0 2011-11-03 11:14:06.481006 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.481027 7fb269f70700 client.5557 3 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:06.481046 7fb269f70700 client.5557 _lookup on #1000000005e/system 2011-11-03 11:14:06.481068 7fb269f70700 client.5557 choose_target_mds starting with req->inode 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.481082 7fb269f70700 client.5557 choose_target_mds inode dir hash is 2 on system => 1733530773 2011-11-03 11:14:06.481103 7fb269f70700 client.5557 choose_target_mds 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) is_hash=1 hash=1733530773 2011-11-03 11:14:06.481124 7fb269f70700 client.5557 choose_target_mds from caps on inode 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.481137 7fb269f70700 client.5557 mds is 0 2011-11-03 11:14:06.481149 7fb269f70700 client.5557 send_request rebuilding request 5 for mds.0 2011-11-03 11:14:06.481162 7fb269f70700 client.5557 encode_cap_releases enter (req: 0x417e6230, mds: 0) 2011-11-03 11:14:06.481178 7fb269f70700 client.5557 send_request set sent_stamp to 2011-11-03 11:14:06.481174 2011-11-03 11:14:06.481191 7fb269f70700 client.5557 send_request client_request(unknown.0:5 lookup #1000000005e/system) v1 to mds.0 2011-11-03 11:14:06.481225 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5557:5 lookup #1000000005e/system) v1 -- ?+0 0x417e5480 2011-11-03 11:14:06.481244 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_request(client.5557:5 lookup #1000000005e/system) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.481264 7fb269f70700 client.5557 awaiting reply|forward|kick on 0x7fb269f6f030 2011-11-03 11:14:06.481293 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.481322 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 9 0x417e5480 client_request(client.5557:5 lookup #1000000005e/system) v1 2011-11-03 11:14:06.481347 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 585 to dispatch throttler 585/104857600 2011-11-03 11:14:06.481371 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x417e6250 2011-11-03 11:14:06.481397 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 9 0x417e5480 2011-11-03 11:14:06.481420 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x417e5480 2011-11-03 11:14:06.481447 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.481464 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.481736 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.481767 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.481789 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.481812 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=26 src mds.0 front=585 data=0 off 0 2011-11-03 11:14:06.481829 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 585 from dispatch throttler 0/104857600 2011-11-03 11:14:06.481856 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 585 2011-11-03 11:14:06.481880 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.481897 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 585 + 0 + 0 byte message 2011-11-03 11:14:06.481922 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 9 0x417e6950 client_reply(???:5 = 0 Success) v1 2011-11-03 11:14:06.481946 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.481969 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.481999 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.482031 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 9 2011-11-03 11:14:06.482059 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.482083 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.482104 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x417e6950 2011-11-03 11:14:06.482126 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 9 ==== client_reply(???:5 = 0 Success) v1 ==== 585+0+0 (919883856 0 0) 0x417e6950 con 0x7fb2709b9a40 2011-11-03 11:14:06.482140 7fb26bb77700 client.5557 handle_client_reply got a reply. Safe:1 tid 5 2011-11-03 11:14:06.482155 7fb26bb77700 client.5557 insert_trace from 2011-11-03 11:14:06.481174 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.482167 7fb26bb77700 client.5557 features 0xffa 2011-11-03 11:14:06.482179 7fb26bb77700 client.5557 update_snap_trace len 48 2011-11-03 11:14:06.482194 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=6 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.482206 7fb26bb77700 client.5557 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.482231 7fb26bb77700 client.5557 add_update_inode adding 100000000af.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41897840) caps pAsLsXs 2011-11-03 11:14:06.482244 7fb26bb77700 client.5557 dir hash is 2 2011-11-03 11:14:06.482261 7fb26bb77700 client.5557 update_inode_file_bits 100000000af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=0.000000 caps=- 0x41897840) - mtime 2011-11-03 11:14:05.781412 2011-11-03 11:14:06.482274 7fb26bb77700 client.5557 add_update_cap first one, opened snaprealm 0x41c33f90 2011-11-03 11:14:06.482301 7fb26bb77700 client.5557 add_update_cap issued - -> pAsLsXs from mds.0 on 100000000af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) 0x41897840) 2011-11-03 11:14:06.482325 7fb26bb77700 client.5557 add_update_inode had 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) caps pAsLsXs 2011-11-03 11:14:06.482351 7fb26bb77700 client.5557 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.482364 7fb26bb77700 client.5557 got dirfrag map for 1000000005e frag * to mds -1 2011-11-03 11:14:06.482381 7fb26bb77700 client.5557 insert_dentry_inode 'system' vino 100000000af.head in dir 1000000005e.head dn 0 2011-11-03 11:14:06.482396 7fb26bb77700 inode.get on 0x41897840 100000000af.head now 1 2011-11-03 11:14:06.482411 7fb26bb77700 client.5557 link dir 0x417e6510 'system' to inode 0x41897840 dn 0x417e5530 (new dn) 2011-11-03 11:14:06.482424 7fb26bb77700 inode.get on 0x41897840 100000000af.head now 2 2011-11-03 11:14:06.482436 7fb26bb77700 client.5557 link inode 0x41897840 parents now 0x417e5530 2011-11-03 11:14:06.482449 7fb26bb77700 inode.put on 0x41897840 100000000af.head now 1 2011-11-03 11:14:06.482461 7fb26bb77700 client.5557 setting dn offset to 2 2011-11-03 11:14:06.482474 7fb26bb77700 client.5557 got dentry lease on system dur 30000ms ttl 2011-11-03 11:14:36.481174 2011-11-03 11:14:06.482487 7fb26bb77700 client.5557 handle_client_reply signalling caller 0x7fb269f6f030 2011-11-03 11:14:06.482505 7fb26bb77700 client.5557 handle_client_reply awaiting kickback on tid 5 0x7fb26bb76490 2011-11-03 11:14:06.482533 7fb269f70700 client.5557 sendrecv kickback on tid 5 0x7fb26bb76490 2011-11-03 11:14:06.482554 7fb269f70700 client.5557 lat 0.001376 2011-11-03 11:14:06.482569 7fb269f70700 client.5557 _lookup res is 0 2011-11-03 11:14:06.482601 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.482637 7fb269f70700 client.5557 4 100000000af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:06.482658 7fb269f70700 client.5557 _lookup on #100000000af/job_201111031113_0001 2011-11-03 11:14:06.482680 7fb269f70700 client.5557 choose_target_mds starting with req->inode 100000000af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.482693 7fb269f70700 client.5557 choose_target_mds inode dir hash is 2 on job_201111031113_0001 => 3184690367 2011-11-03 11:14:06.482715 7fb269f70700 client.5557 choose_target_mds 100000000af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) is_hash=1 hash=3184690367 2011-11-03 11:14:06.482736 7fb269f70700 client.5557 choose_target_mds from caps on inode 100000000af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.482748 7fb269f70700 client.5557 mds is 0 2011-11-03 11:14:06.482761 7fb269f70700 client.5557 send_request rebuilding request 6 for mds.0 2011-11-03 11:14:06.482774 7fb269f70700 client.5557 encode_cap_releases enter (req: 0x41898290, mds: 0) 2011-11-03 11:14:06.482789 7fb269f70700 client.5557 send_request set sent_stamp to 2011-11-03 11:14:06.482785 2011-11-03 11:14:06.482802 7fb269f70700 client.5557 send_request client_request(unknown.0:6 lookup #100000000af/job_201111031113_0001) v1 to mds.0 2011-11-03 11:14:06.482821 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5557:6 lookup #100000000af/job_201111031113_0001) v1 -- ?+0 0x41898600 2011-11-03 11:14:06.482839 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_request(client.5557:6 lookup #100000000af/job_201111031113_0001) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.482859 7fb269f70700 client.5557 awaiting reply|forward|kick on 0x7fb269f6f030 2011-11-03 11:14:06.482888 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.482914 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 10 0x41898600 client_request(client.5557:6 lookup #100000000af/job_201111031113_0001) v1 2011-11-03 11:14:06.482937 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 10 0x41898600 2011-11-03 11:14:06.482954 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x41898600 2011-11-03 11:14:06.482981 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.482998 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.483020 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 585 to dispatch throttler 585/104857600 2011-11-03 11:14:06.483034 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x417e6950 2011-11-03 11:14:06.483318 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.483350 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.483379 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.483412 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=26 src mds.0 front=596 data=0 off 0 2011-11-03 11:14:06.483432 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 596 from dispatch throttler 0/104857600 2011-11-03 11:14:06.483456 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 596 2011-11-03 11:14:06.483479 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.483496 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 596 + 0 + 0 byte message 2011-11-03 11:14:06.483526 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 10 0x417e6230 client_reply(???:6 = 0 Success) v1 2011-11-03 11:14:06.483549 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.483577 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.483605 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.483630 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 10 2011-11-03 11:14:06.483689 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x417e6230 2011-11-03 11:14:06.483723 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 10 ==== client_reply(???:6 = 0 Success) v1 ==== 596+0+0 (1706751312 0 0) 0x417e6230 con 0x7fb2709b9a40 2011-11-03 11:14:06.483741 7fb26bb77700 client.5557 handle_client_reply got a reply. Safe:1 tid 6 2011-11-03 11:14:06.483760 7fb26bb77700 client.5557 insert_trace from 2011-11-03 11:14:06.482785 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.483776 7fb26bb77700 client.5557 features 0xffa 2011-11-03 11:14:06.483793 7fb26bb77700 client.5557 update_snap_trace len 48 2011-11-03 11:14:06.483812 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=7 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.483829 7fb26bb77700 client.5557 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.483855 7fb26bb77700 client.5557 add_update_inode adding 100000000b1.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41898600) caps pAsLs 2011-11-03 11:14:06.483876 7fb26bb77700 client.5557 dir hash is 2 2011-11-03 11:14:06.483897 7fb26bb77700 client.5557 update_inode_file_bits 100000000b1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x41898600) - mtime 2011-11-03 11:14:05.785578 2011-11-03 11:14:06.483915 7fb26bb77700 client.5557 add_update_cap first one, opened snaprealm 0x41c33f90 2011-11-03 11:14:06.483945 7fb26bb77700 client.5557 add_update_cap issued - -> pAsLs from mds.0 on 100000000b1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) 0x41898600) 2011-11-03 11:14:06.483973 7fb26bb77700 client.5557 add_update_inode had 100000000af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) caps pAsLsXs 2011-11-03 11:14:06.484009 7fb26bb77700 client.5557 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 100000000af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.484041 7fb26bb77700 client.5557 got dirfrag map for 100000000af frag * to mds -1 2011-11-03 11:14:06.484063 7fb26bb77700 client.5557 insert_dentry_inode 'job_201111031113_0001' vino 100000000b1.head in dir 100000000af.head dn 0 2011-11-03 11:14:06.484081 7fb26bb77700 inode.get on 0x41898600 100000000b1.head now 1 2011-11-03 11:14:06.484099 7fb26bb77700 client.5557 link dir 0x41897840 'job_201111031113_0001' to inode 0x41898600 dn 0x41c358a0 (new dn) 2011-11-03 11:14:06.484116 7fb26bb77700 inode.get on 0x41898600 100000000b1.head now 2 2011-11-03 11:14:06.484134 7fb26bb77700 client.5557 link inode 0x41898600 parents now 0x41c358a0 2011-11-03 11:14:06.484154 7fb26bb77700 inode.put on 0x41898600 100000000b1.head now 1 2011-11-03 11:14:06.484171 7fb26bb77700 client.5557 setting dn offset to 2 2011-11-03 11:14:06.484188 7fb26bb77700 client.5557 got dentry lease on job_201111031113_0001 dur 30000ms ttl 2011-11-03 11:14:36.482785 2011-11-03 11:14:06.484206 7fb26bb77700 client.5557 handle_client_reply signalling caller 0x7fb269f6f030 2011-11-03 11:14:06.484227 7fb26bb77700 client.5557 handle_client_reply awaiting kickback on tid 6 0x7fb26bb76490 2011-11-03 11:14:06.484254 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.484280 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.484302 7fb269f70700 client.5557 sendrecv kickback on tid 6 0x7fb26bb76490 2011-11-03 11:14:06.484323 7fb269f70700 client.5557 lat 0.001535 2011-11-03 11:14:06.484339 7fb269f70700 client.5557 _lookup res is 0 2011-11-03 11:14:06.484370 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.484391 7fb269f70700 client.5557 5 100000000b1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:06.484410 7fb269f70700 client.5557 _lookup on #100000000b1/jobToken 2011-11-03 11:14:06.484432 7fb269f70700 client.5557 choose_target_mds starting with req->inode 100000000b1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.484446 7fb269f70700 client.5557 choose_target_mds inode dir hash is 2 on jobToken => 3041680973 2011-11-03 11:14:06.484467 7fb269f70700 client.5557 choose_target_mds 100000000b1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) is_hash=1 hash=3041680973 2011-11-03 11:14:06.484487 7fb269f70700 client.5557 choose_target_mds from caps on inode 100000000b1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.484499 7fb269f70700 client.5557 mds is 0 2011-11-03 11:14:06.484512 7fb269f70700 client.5557 send_request rebuilding request 7 for mds.0 2011-11-03 11:14:06.484525 7fb269f70700 client.5557 encode_cap_releases enter (req: 0x417e6230, mds: 0) 2011-11-03 11:14:06.484541 7fb269f70700 client.5557 send_request set sent_stamp to 2011-11-03 11:14:06.484537 2011-11-03 11:14:06.484555 7fb269f70700 client.5557 send_request client_request(unknown.0:7 lookup #100000000b1/jobToken) v1 to mds.0 2011-11-03 11:14:06.484573 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5557:7 lookup #100000000b1/jobToken) v1 -- ?+0 0x417e5770 2011-11-03 11:14:06.484592 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_request(client.5557:7 lookup #100000000b1/jobToken) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.484624 7fb269f70700 client.5557 awaiting reply|forward|kick on 0x7fb269f6f030 2011-11-03 11:14:06.484652 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.484681 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 11 0x417e5770 client_request(client.5557:7 lookup #100000000b1/jobToken) v1 2011-11-03 11:14:06.484707 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 11 0x417e5770 2011-11-03 11:14:06.484729 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x417e5770 2011-11-03 11:14:06.484763 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 596 to dispatch throttler 596/104857600 2011-11-03 11:14:06.484786 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x417e6230 2011-11-03 11:14:06.484813 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.484835 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.484966 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.484996 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.485019 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.485042 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=26 src mds.0 front=583 data=0 off 0 2011-11-03 11:14:06.485059 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 583 from dispatch throttler 0/104857600 2011-11-03 11:14:06.485083 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 583 2011-11-03 11:14:06.485106 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.485122 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 583 + 0 + 0 byte message 2011-11-03 11:14:06.485150 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 11 0x41899050 client_reply(???:7 = 0 Success) v1 2011-11-03 11:14:06.485173 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.485196 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.485225 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.485248 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 11 2011-11-03 11:14:06.485277 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x41899050 2011-11-03 11:14:06.485307 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 11 ==== client_reply(???:7 = 0 Success) v1 ==== 583+0+0 (748133898 0 0) 0x41899050 con 0x7fb2709b9a40 2011-11-03 11:14:06.485345 7fb26bb77700 client.5557 handle_client_reply got a reply. Safe:1 tid 7 2011-11-03 11:14:06.485366 7fb26bb77700 client.5557 insert_trace from 2011-11-03 11:14:06.484537 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.485383 7fb26bb77700 client.5557 features 0xffa 2011-11-03 11:14:06.485399 7fb26bb77700 client.5557 update_snap_trace len 48 2011-11-03 11:14:06.485418 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=8 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.485434 7fb26bb77700 client.5557 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.485461 7fb26bb77700 client.5557 add_update_inode adding 100000000b2.head(ref=0 cap_refs={} open={} mode=100000 size=0 mtime=0.000000 caps=- objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] 0x41899360) caps pLs 2011-11-03 11:14:06.485484 7fb26bb77700 client.5557 update_inode_file_bits 100000000b2.head(ref=0 cap_refs={} open={} mode=100777 size=0 mtime=0.000000 caps=- objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] 0x41899360) - mtime 2011-11-03 11:14:05.785578 2011-11-03 11:14:06.485502 7fb26bb77700 client.5557 add_update_cap first one, opened snaprealm 0x41c33f90 2011-11-03 11:14:06.485531 7fb26bb77700 client.5557 add_update_cap issued - -> pLs from mds.0 on 100000000b2.head(ref=0 cap_refs={} open={} mode=100777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pLs(0=pLs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] 0x41899360) 2011-11-03 11:14:06.485558 7fb26bb77700 client.5557 add_update_inode had 100000000b1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) caps pAsLs 2011-11-03 11:14:06.485588 7fb26bb77700 client.5557 add_update_cap issued pAsLs -> pAsLs from mds.0 on 100000000b1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.485605 7fb26bb77700 client.5557 got dirfrag map for 100000000b1 frag * to mds -1 2011-11-03 11:14:06.485626 7fb26bb77700 client.5557 insert_dentry_inode 'jobToken' vino 100000000b2.head in dir 100000000b1.head dn 0 2011-11-03 11:14:06.485643 7fb26bb77700 inode.get on 0x41899360 100000000b2.head now 1 2011-11-03 11:14:06.485661 7fb26bb77700 client.5557 link dir 0x41898600 'jobToken' to inode 0x41899360 dn 0x417e6b80 (new dn) 2011-11-03 11:14:06.485678 7fb26bb77700 inode.get on 0x41899360 100000000b2.head now 2 2011-11-03 11:14:06.485695 7fb26bb77700 client.5557 link inode 0x41899360 parents now 0x417e6b80 2011-11-03 11:14:06.485712 7fb26bb77700 inode.put on 0x41899360 100000000b2.head now 1 2011-11-03 11:14:06.485728 7fb26bb77700 client.5557 setting dn offset to 2 2011-11-03 11:14:06.485746 7fb26bb77700 client.5557 got dentry lease on jobToken dur 30000ms ttl 2011-11-03 11:14:36.484537 2011-11-03 11:14:06.485764 7fb26bb77700 client.5557 handle_client_reply signalling caller 0x7fb269f6f030 2011-11-03 11:14:06.485784 7fb26bb77700 client.5557 handle_client_reply awaiting kickback on tid 7 0x7fb26bb76490 2011-11-03 11:14:06.485812 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.485838 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.485860 7fb269f70700 client.5557 sendrecv kickback on tid 7 0x7fb26bb76490 2011-11-03 11:14:06.485882 7fb269f70700 client.5557 lat 0.001342 2011-11-03 11:14:06.485897 7fb269f70700 client.5557 _lookup res is 0 2011-11-03 11:14:06.485928 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={} open={} mode=100777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pLs(0=pLs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.485970 7fb269f70700 client.5557 _getattr mask pAsLsXsFs issued=0 2011-11-03 11:14:06.485995 7fb269f70700 client.5557 choose_target_mds starting with req->inode 100000000b2.head(ref=1 cap_refs={} open={} mode=100777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pLs(0=pLs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.486017 7fb269f70700 client.5557 choose_target_mds 100000000b2.head(ref=1 cap_refs={} open={} mode=100777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pLs(0=pLs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) is_hash=0 hash=0 2011-11-03 11:14:06.486037 7fb269f70700 client.5557 choose_target_mds from caps on inode 100000000b2.head(ref=1 cap_refs={} open={} mode=100777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pLs(0=pLs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.486049 7fb269f70700 client.5557 mds is 0 2011-11-03 11:14:06.486061 7fb269f70700 client.5557 send_request rebuilding request 8 for mds.0 2011-11-03 11:14:06.486074 7fb269f70700 client.5557 encode_cap_releases enter (req: 0x418997a0, mds: 0) 2011-11-03 11:14:06.486090 7fb269f70700 client.5557 send_request set sent_stamp to 2011-11-03 11:14:06.486086 2011-11-03 11:14:06.486106 7fb269f70700 client.5557 send_request client_request(unknown.0:8 getattr pAsLsXsFs #100000000b2) v1 to mds.0 2011-11-03 11:14:06.486127 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5557:8 getattr pAsLsXsFs #100000000b2) v1 -- ?+0 0x41898290 2011-11-03 11:14:06.486148 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_request(client.5557:8 getattr pAsLsXsFs #100000000b2) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.486166 7fb269f70700 client.5557 awaiting reply|forward|kick on 0x7fb269f6f250 2011-11-03 11:14:06.486194 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.486227 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 12 0x41898290 client_request(client.5557:8 getattr pAsLsXsFs #100000000b2) v1 2011-11-03 11:14:06.486254 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 12 0x41898290 2011-11-03 11:14:06.486275 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x41898290 2011-11-03 11:14:06.486310 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 583 to dispatch throttler 583/104857600 2011-11-03 11:14:06.486333 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x41899050 2011-11-03 11:14:06.486361 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.486382 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.543875 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.543912 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.543935 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.543959 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=26 src mds.0 front=316 data=0 off 0 2011-11-03 11:14:06.543975 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 316 from dispatch throttler 0/104857600 2011-11-03 11:14:06.544014 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 316 2011-11-03 11:14:06.544037 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.544054 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 316 + 0 + 0 byte message 2011-11-03 11:14:06.544081 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 12 0x417e6230 client_reply(???:8 = 0 Success) v1 2011-11-03 11:14:06.544104 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.544128 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.544161 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.544188 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 12 2011-11-03 11:14:06.544217 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x417e6230 2011-11-03 11:14:06.544246 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 12 ==== client_reply(???:8 = 0 Success) v1 ==== 316+0+0 (1566958759 0 0) 0x417e6230 con 0x7fb2709b9a40 2011-11-03 11:14:06.544265 7fb26bb77700 client.5557 handle_client_reply got a reply. Safe:1 tid 8 2011-11-03 11:14:06.544283 7fb26bb77700 client.5557 insert_trace from 2011-11-03 11:14:06.486086 mds.0 is_target=1 is_dentry=0 2011-11-03 11:14:06.544300 7fb26bb77700 client.5557 features 0xffa 2011-11-03 11:14:06.544316 7fb26bb77700 client.5557 update_snap_trace len 48 2011-11-03 11:14:06.544334 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=9 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.544351 7fb26bb77700 client.5557 hrm is_target=1 is_dentry=0 2011-11-03 11:14:06.544381 7fb26bb77700 client.5557 add_update_inode had 100000000b2.head(ref=1 cap_refs={} open={} mode=100777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pLs(0=pLs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) caps pAsLsXs 2011-11-03 11:14:06.544412 7fb26bb77700 client.5557 update_inode_file_bits 100000000b2.head(ref=1 cap_refs={} open={} mode=100777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pLs(0=pLs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) pLs mtime 2011-11-03 11:14:05.788394 2011-11-03 11:14:06.544429 7fb26bb77700 client.5557 size 0 -> 106 2011-11-03 11:14:06.544461 7fb26bb77700 client.5557 add_update_cap issued pLs -> pAsLsXs from mds.0 on 100000000b2.head(ref=1 cap_refs={} open={} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXs(0=pAsLsXs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.544479 7fb26bb77700 client.5557 handle_client_reply signalling caller 0x7fb269f6f250 2011-11-03 11:14:06.544500 7fb26bb77700 client.5557 handle_client_reply awaiting kickback on tid 8 0x7fb26bb76490 2011-11-03 11:14:06.544529 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.544554 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.544581 7fb269f70700 client.5557 sendrecv kickback on tid 8 0x7fb26bb76490 2011-11-03 11:14:06.544610 7fb269f70700 client.5557 lat 0.058521 2011-11-03 11:14:06.544635 7fb269f70700 client.5557 _getattr result=0 2011-11-03 11:14:06.544656 7fb269f70700 client.5557 fill_stat on 100000000b2 snap/devhead mode 0100777 mtime 2011-11-03 11:14:05.788394 ctime 2011-11-03 11:14:05.785578 2011-11-03 11:14:06.544672 7fb269f70700 client.5557 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:06.544704 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 316 to dispatch throttler 316/104857600 2011-11-03 11:14:06.544724 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x417e6230 2011-11-03 11:14:06.544744 7fb269f70700 client.5557 open enter(/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0,0) = 2011-11-03 11:14:06.544768 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:06.544792 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:06.544810 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.544851 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.544874 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:06.544889 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.544924 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.544946 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:06.544960 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.544995 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.545016 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:06.545032 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:06.545067 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.545089 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:06.545104 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:06.545135 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.545175 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:06.545193 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:06.545225 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={} open={} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXs(0=pAsLsXs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.545269 7fb269f70700 client.5557 choose_target_mds starting with req->inode 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXs(0=pAsLsXs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.545294 7fb269f70700 client.5557 choose_target_mds 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXs(0=pAsLsXs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) is_hash=0 hash=0 2011-11-03 11:14:06.545316 7fb269f70700 client.5557 choose_target_mds from caps on inode 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXs(0=pAsLsXs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.545329 7fb269f70700 client.5557 mds is 0 2011-11-03 11:14:06.545341 7fb269f70700 client.5557 send_request rebuilding request 9 for mds.0 2011-11-03 11:14:06.545355 7fb269f70700 client.5557 encode_cap_releases enter (req: 0x417e6230, mds: 0) 2011-11-03 11:14:06.545370 7fb269f70700 client.5557 send_request set sent_stamp to 2011-11-03 11:14:06.545365 2011-11-03 11:14:06.545383 7fb269f70700 client.5557 send_request client_request(unknown.0:9 open #100000000b2) v1 to mds.0 2011-11-03 11:14:06.545402 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5557:9 open #100000000b2) v1 -- ?+0 0x41898290 2011-11-03 11:14:06.545421 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_request(client.5557:9 open #100000000b2) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.545438 7fb269f70700 client.5557 awaiting reply|forward|kick on 0x7fb269f6f270 2011-11-03 11:14:06.545466 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.545493 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 13 0x41898290 client_request(client.5557:9 open #100000000b2) v1 2011-11-03 11:14:06.545516 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 13 0x41898290 2011-11-03 11:14:06.545533 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x41898290 2011-11-03 11:14:06.545560 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.545577 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.545752 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.545798 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.545832 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.545857 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=784 src mds.0 front=180 data=0 off 0 2011-11-03 11:14:06.545873 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 180 from dispatch throttler 0/104857600 2011-11-03 11:14:06.545897 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 180 2011-11-03 11:14:06.545920 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.545936 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 180 + 0 + 0 byte message 2011-11-03 11:14:06.545984 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 13 0x418997a0 client_caps(grant ino 100000000b2 6752 seq 3 caps=pAsLsXsFcb dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v2 2011-11-03 11:14:06.546008 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.546031 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.546061 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.546087 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 13 2011-11-03 11:14:06.546118 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.546151 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x418997a0 2011-11-03 11:14:06.546194 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 13 ==== client_caps(grant ino 100000000b2 6752 seq 3 caps=pAsLsXsFcb dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v2 ==== 180+0+0 (4011990669 0 0) 0x418997a0 con 0x7fb2709b9a40 2011-11-03 11:14:06.546225 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=26 src mds.0 front=312 data=0 off 0 2011-11-03 11:14:06.546250 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 312 from dispatch throttler 180/104857600 2011-11-03 11:14:06.546273 7fb26bb77700 client.5557 mds.0 seq now 1 2011-11-03 11:14:06.546305 7fb26bb77700 client.5557 handle_cap_grant on in 100000000b2 mds.0 seq 3 caps now pAsLsXsFcb was pAsLsXs 2011-11-03 11:14:06.546342 7fb26bb77700 client.5557 update_inode_file_bits 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXs(0=pAsLsXs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) pAsLsXs mtime 2011-11-03 11:14:05.788394 2011-11-03 11:14:06.546364 7fb26bb77700 client.5557 grant, new caps are Fcb 2011-11-03 11:14:06.546386 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 180 to dispatch throttler 492/104857600 2011-11-03 11:14:06.546404 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x418997a0 2011-11-03 11:14:06.546447 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 312 2011-11-03 11:14:06.546482 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.546507 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.546537 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.546559 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 312 + 0 + 0 byte message 2011-11-03 11:14:06.546587 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 14 0x41899050 client_reply(???:9 = 0 Success) v1 2011-11-03 11:14:06.546611 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.546634 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.546662 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.546686 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 14 2011-11-03 11:14:06.546716 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x41899050 2011-11-03 11:14:06.546746 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 14 ==== client_reply(???:9 = 0 Success) v1 ==== 312+0+0 (2607006670 0 0) 0x41899050 con 0x7fb2709b9a40 2011-11-03 11:14:06.546764 7fb26bb77700 client.5557 handle_client_reply got a reply. Safe:1 tid 9 2011-11-03 11:14:06.546783 7fb26bb77700 client.5557 insert_trace from 2011-11-03 11:14:06.545365 mds.0 is_target=1 is_dentry=0 2011-11-03 11:14:06.546799 7fb26bb77700 client.5557 features 0xffa 2011-11-03 11:14:06.546815 7fb26bb77700 client.5557 update_snap_trace len 48 2011-11-03 11:14:06.546834 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=9 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.546850 7fb26bb77700 client.5557 hrm is_target=1 is_dentry=0 2011-11-03 11:14:06.546883 7fb26bb77700 client.5557 add_update_inode had 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) caps pAsLsXsFcb 2011-11-03 11:14:06.546918 7fb26bb77700 client.5557 update_inode_file_bits 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) pAsLsXsFcb mtime 2011-11-03 11:14:05.788394 2011-11-03 11:14:06.546955 7fb26bb77700 client.5557 add_update_cap issued pAsLsXsFcb -> pAsLsXsFcb from mds.0 on 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.546973 7fb26bb77700 client.5557 handle_client_reply signalling caller 0x7fb269f6f270 2011-11-03 11:14:06.546994 7fb26bb77700 client.5557 handle_client_reply awaiting kickback on tid 9 0x7fb26bb76490 2011-11-03 11:14:06.547022 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.547052 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.547085 7fb269f70700 client.5557 sendrecv kickback on tid 9 0x7fb26bb76490 2011-11-03 11:14:06.547107 7fb269f70700 client.5557 lat 0.001739 2011-11-03 11:14:06.547130 7fb269f70700 inode.get on 0x41899360 100000000b2.head now 2 2011-11-03 11:14:06.547143 7fb269f70700 client.5557 _create_fh 100000000b2 mode 1 2011-11-03 11:14:06.547158 7fb269f70700 client.5557 trim_cache size 6 max 16384 2011-11-03 11:14:06.547182 7fb269f70700 client.5557 open exit(#1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0) = 10 2011-11-03 11:14:06.547205 7fb269f70700 client.5557 describe_layout(10) = 0 2011-11-03 11:14:06.547231 7fb269f70700 client.5557 close enter(10) 2011-11-03 11:14:06.547265 7fb269f70700 client.5557 _release_fh 0x417e6c80 mode 1 on 100000000b2.head(ref=2 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.547300 7fb269f70700 client.5557 _flush 100000000b2.head(ref=2 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.547318 7fb269f70700 client.5557 nothing to flush 2011-11-03 11:14:06.547352 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=2 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted - used - is_delayed=0 2011-11-03 11:14:06.547383 7fb269f70700 client.5557 cap_delay_requeue on 100000000b2.head(ref=2 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.547413 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFcb implemented pAsLsXsFcb revoking - 2011-11-03 11:14:06.547431 7fb269f70700 client.5557 delaying cap release 2011-11-03 11:14:06.547461 7fb269f70700 client.5557 put_inode on 100000000b2.head(ref=2 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.547481 7fb269f70700 inode.put on 0x41899360 100000000b2.head now 1 2011-11-03 11:14:06.547500 7fb269f70700 client.5557 close exit(10) 2011-11-03 11:14:06.547529 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 312 to dispatch throttler 312/104857600 2011-11-03 11:14:06.547550 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x41899050 2011-11-03 11:14:06.553754 7fb269f70700 client.5557 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:06.553823 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:06.553859 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:06.553878 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.553917 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.553940 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:06.553967 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.554017 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.554040 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:06.554054 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.554089 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.554111 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:06.554126 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:06.554159 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.554181 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:06.554196 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:06.554227 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.554248 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:06.554263 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:06.554297 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.554319 7fb269f70700 client.5557 _getattr mask pAsLsXsFs issued=0 2011-11-03 11:14:06.554350 7fb269f70700 client.5557 choose_target_mds starting with req->inode 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.554375 7fb269f70700 client.5557 choose_target_mds 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) is_hash=0 hash=0 2011-11-03 11:14:06.554399 7fb269f70700 client.5557 choose_target_mds from caps on inode 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.554424 7fb269f70700 client.5557 mds is 0 2011-11-03 11:14:06.554438 7fb269f70700 client.5557 send_request rebuilding request 10 for mds.0 2011-11-03 11:14:06.554454 7fb269f70700 client.5557 encode_cap_releases enter (req: 0x417e6230, mds: 0) 2011-11-03 11:14:06.554470 7fb269f70700 client.5557 send_request set sent_stamp to 2011-11-03 11:14:06.554466 2011-11-03 11:14:06.554488 7fb269f70700 client.5557 send_request client_request(unknown.0:10 getattr pAsLsXsFs #100000000b2) v1 to mds.0 2011-11-03 11:14:06.554515 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5557:10 getattr pAsLsXsFs #100000000b2) v1 -- ?+0 0x417e5770 2011-11-03 11:14:06.554540 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_request(client.5557:10 getattr pAsLsXsFs #100000000b2) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.554569 7fb269f70700 client.5557 awaiting reply|forward|kick on 0x7fb269f6f060 2011-11-03 11:14:06.554601 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.554633 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 14 0x417e5770 client_request(client.5557:10 getattr pAsLsXsFs #100000000b2) v1 2011-11-03 11:14:06.554656 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 14 0x417e5770 2011-11-03 11:14:06.554673 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x417e5770 2011-11-03 11:14:06.554702 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.554718 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.679084 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.679121 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.679143 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.679167 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=26 src mds.0 front=312 data=0 off 0 2011-11-03 11:14:06.679184 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 312 from dispatch throttler 0/104857600 2011-11-03 11:14:06.679208 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 312 2011-11-03 11:14:06.679231 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.679248 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 312 + 0 + 0 byte message 2011-11-03 11:14:06.679274 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 15 0x41899050 client_reply(???:10 = 0 Success) v1 2011-11-03 11:14:06.679298 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.679337 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.679368 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.679396 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 15 2011-11-03 11:14:06.679423 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.679455 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x41899050 2011-11-03 11:14:06.679488 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 15 ==== client_reply(???:10 = 0 Success) v1 ==== 312+0+0 (298254053 0 0) 0x41899050 con 0x7fb2709b9a40 2011-11-03 11:14:06.679518 7fb26bb77700 client.5557 handle_client_reply got a reply. Safe:1 tid 10 2011-11-03 11:14:06.679537 7fb26bb77700 client.5557 insert_trace from 2011-11-03 11:14:06.554466 mds.0 is_target=1 is_dentry=0 2011-11-03 11:14:06.679553 7fb26bb77700 client.5557 features 0xffa 2011-11-03 11:14:06.679569 7fb26bb77700 client.5557 update_snap_trace len 48 2011-11-03 11:14:06.679589 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=9 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.679606 7fb26bb77700 client.5557 hrm is_target=1 is_dentry=0 2011-11-03 11:14:06.679643 7fb26bb77700 client.5557 add_update_inode had 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) caps pAsLsXsFscrb 2011-11-03 11:14:06.679677 7fb26bb77700 client.5557 update_inode_file_bits 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) pAsLsXsFcb mtime 2011-11-03 11:14:05.788394 2011-11-03 11:14:06.679713 7fb26bb77700 client.5557 add_update_cap issued pAsLsXsFcb -> pAsLsXsFscrb from mds.0 on 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.679732 7fb26bb77700 client.5557 handle_client_reply signalling caller 0x7fb269f6f060 2011-11-03 11:14:06.679753 7fb26bb77700 client.5557 handle_client_reply awaiting kickback on tid 10 0x7fb26bb76490 2011-11-03 11:14:06.679783 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.679808 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.679837 7fb269f70700 client.5557 sendrecv kickback on tid 10 0x7fb26bb76490 2011-11-03 11:14:06.679865 7fb269f70700 client.5557 lat 0.125395 2011-11-03 11:14:06.679896 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=784 src mds.0 front=180 data=0 off 0 2011-11-03 11:14:06.679924 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 180 from dispatch throttler 312/104857600 2011-11-03 11:14:06.679952 7fb269f70700 client.5557 _getattr result=0 2011-11-03 11:14:06.679978 7fb269f70700 client.5557 fill_stat on 100000000b2 snap/devhead mode 0100777 mtime 2011-11-03 11:14:05.788394 ctime 2011-11-03 11:14:05.785578 2011-11-03 11:14:06.679998 7fb269f70700 client.5557 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:06.680039 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 180 2011-11-03 11:14:06.680077 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.680099 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 180 + 0 + 0 byte message 2011-11-03 11:14:06.680127 7fb269f70700 client.5557 open enter(/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0,0) = 2011-11-03 11:14:06.680156 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:06.680185 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:06.680203 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.680231 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 312 to dispatch throttler 492/104857600 2011-11-03 11:14:06.680253 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x41899050 2011-11-03 11:14:06.680300 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.680331 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:06.680350 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.680390 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 16 0x418997a0 client_caps(grant ino 100000000b2 6752 seq 5 caps=pAsLsXsFscrb dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v2 2011-11-03 11:14:06.680421 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.680449 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.680479 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.680509 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 16 2011-11-03 11:14:06.680533 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.680569 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=784 src mds.0 front=180 data=0 off 0 2011-11-03 11:14:06.680590 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 180 from dispatch throttler 180/104857600 2011-11-03 11:14:06.680620 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 180 2011-11-03 11:14:06.680646 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x418997a0 2011-11-03 11:14:06.680686 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 16 ==== client_caps(grant ino 100000000b2 6752 seq 5 caps=pAsLsXsFscrb dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v2 ==== 180+0+0 (979550186 0 0) 0x418997a0 con 0x7fb2709b9a40 2011-11-03 11:14:06.680726 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.680751 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 180 + 0 + 0 byte message 2011-11-03 11:14:06.680798 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.680831 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:06.680850 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.680892 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.680918 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:06.680938 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:06.680976 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.681002 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:06.681023 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:06.681060 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.681085 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:06.681105 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:06.681145 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.681174 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.681199 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.681243 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted pFscr used - is_delayed=1 2011-11-03 11:14:06.681287 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFscrb implemented pAsLsXsFscrb revoking - 2011-11-03 11:14:06.681325 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 17 0x4195c710 client_caps(revoke ino 100000000b2 6752 seq 6 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v2 2011-11-03 11:14:06.681356 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.681381 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.681438 7fb269f70700 client.5557 send_cap 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) mds.0 seq 5 used - want pFscr flush - retain pAsxLsxXsxFsxcrwbl held pAsLsXsFscrb revoking - dropping - 2011-11-03 11:14:06.681462 7fb269f70700 client.5557 auth cap, setting max_size = 0 2011-11-03 11:14:06.681496 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 100000000b2 6752 seq 5 caps=pAsLsXsFscrb dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 -- ?+0 0x4195d5a0 2011-11-03 11:14:06.681523 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.681550 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 17 2011-11-03 11:14:06.681589 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_caps(update ino 100000000b2 6752 seq 5 caps=pAsLsXsFscrb dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.681613 7fb269f70700 inode.get on 0x41899360 100000000b2.head now 2 2011-11-03 11:14:06.681632 7fb269f70700 client.5557 _create_fh 100000000b2 mode 1 2011-11-03 11:14:06.681651 7fb269f70700 client.5557 trim_cache size 6 max 16384 2011-11-03 11:14:06.681675 7fb269f70700 client.5557 open exit(#1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0) = 11 2011-11-03 11:14:06.681697 7fb269f70700 client.5557 describe_layout(11) = 0 2011-11-03 11:14:06.681717 7fb269f70700 client.5557 close enter(11) 2011-11-03 11:14:06.681755 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 15 0x4195d5a0 client_caps(update ino 100000000b2 6752 seq 5 caps=pAsLsXsFscrb dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 2011-11-03 11:14:06.681785 7fb26bb77700 client.5557 mds.0 seq now 2 2011-11-03 11:14:06.681818 7fb26bb77700 client.5557 handle_cap_grant on in 100000000b2 mds.0 seq 5 caps now pAsLsXsFscrb was pAsLsXsFscrb 2011-11-03 11:14:06.681855 7fb26bb77700 client.5557 update_inode_file_bits 100000000b2.head(ref=2 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) pAsLsXsFscrb mtime 2011-11-03 11:14:05.788394 2011-11-03 11:14:06.681878 7fb26bb77700 client.5557 caps unchanged at pAsLsXsFscrb 2011-11-03 11:14:06.681906 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 15 0x4195d5a0 2011-11-03 11:14:06.681932 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x4195d5a0 2011-11-03 11:14:06.682005 7fb269f70700 client.5557 _release_fh 0x41a54a30 mode 1 on 100000000b2.head(ref=2 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.682045 7fb269f70700 client.5557 _flush 100000000b2.head(ref=2 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.682063 7fb269f70700 client.5557 nothing to flush 2011-11-03 11:14:06.682093 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=2 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted - used - is_delayed=0 2011-11-03 11:14:06.682123 7fb269f70700 client.5557 cap_delay_requeue on 100000000b2.head(ref=2 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.682151 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFscrb implemented pAsLsXsFscrb revoking - 2011-11-03 11:14:06.682168 7fb269f70700 client.5557 delaying cap release 2011-11-03 11:14:06.682196 7fb269f70700 client.5557 put_inode on 100000000b2.head(ref=2 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.682215 7fb269f70700 inode.put on 0x41899360 100000000b2.head now 1 2011-11-03 11:14:06.682233 7fb269f70700 client.5557 close exit(11) 2011-11-03 11:14:06.682261 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.682286 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.682311 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 180 to dispatch throttler 360/104857600 2011-11-03 11:14:06.682332 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x418997a0 2011-11-03 11:14:06.682347 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x4195c710 2011-11-03 11:14:06.682375 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 17 ==== client_caps(revoke ino 100000000b2 6752 seq 6 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v2 ==== 180+0+0 (3291382497 0 0) 0x4195c710 con 0x7fb2709b9a40 2011-11-03 11:14:06.682390 7fb269f70700 client.5557 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:06.682420 7fb26bb77700 client.5557 mds.0 seq now 3 2011-11-03 11:14:06.682444 7fb26bb77700 client.5557 handle_cap_grant on in 100000000b2 mds.0 seq 6 caps now pAsLsXsFscr was pAsLsXsFscrb 2011-11-03 11:14:06.682474 7fb26bb77700 client.5557 update_inode_file_bits 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) pAsLsXsFscrb mtime 2011-11-03 11:14:05.788394 2011-11-03 11:14:06.682490 7fb26bb77700 client.5557 revocation of Fb 2011-11-03 11:14:06.682516 7fb26bb77700 client.5557 check_caps on 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted - used - is_delayed=1 2011-11-03 11:14:06.682543 7fb26bb77700 client.5557 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscrb revoking Fb 2011-11-03 11:14:06.682565 7fb26bb77700 client.5557 completed revocation of Fb 2011-11-03 11:14:06.682598 7fb26bb77700 client.5557 send_cap 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) mds.0 seq 6 used - want - flush - retain pAsLsXsFs held pAsLsXsFscrb revoking Fb dropping Fcr 2011-11-03 11:14:06.682613 7fb26bb77700 client.5557 auth cap, setting max_size = 0 2011-11-03 11:14:06.682640 7fb26bb77700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 100000000b2 6752 seq 6 caps=pAsLsXsFs dirty=- wanted=- follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 -- ?+0 0x418997a0 2011-11-03 11:14:06.682668 7fb26bb77700 -- 192.168.141.130:0/1011511 submit_message client_caps(update ino 100000000b2 6752 seq 6 caps=pAsLsXsFs dirty=- wanted=- follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.682695 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 180 to dispatch throttler 180/104857600 2011-11-03 11:14:06.682715 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x4195c710 2011-11-03 11:14:06.682746 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:06.682779 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:06.682798 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.682840 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.682868 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:06.682887 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.682929 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.682956 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:06.682975 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.683016 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.683043 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:06.683063 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:06.683101 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.683144 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:06.683164 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:06.683201 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.683227 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:06.683246 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:06.683286 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFs(0=pAsLsXsFs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.683308 7fb269f70700 client.5557 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:06.683329 7fb269f70700 client.5557 fill_stat on 100000000b2 snap/devhead mode 0100777 mtime 2011-11-03 11:14:05.788394 ctime 2011-11-03 11:14:05.785578 2011-11-03 11:14:06.683349 7fb269f70700 client.5557 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:06.683377 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.683413 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 16 0x418997a0 client_caps(update ino 100000000b2 6752 seq 6 caps=pAsLsXsFs dirty=- wanted=- follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 2011-11-03 11:14:06.683436 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 16 0x418997a0 2011-11-03 11:14:06.683453 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x418997a0 2011-11-03 11:14:06.683474 7fb269f70700 client.5557 open enter(/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0,0) = 2011-11-03 11:14:06.683505 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:06.683534 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:06.683553 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.683595 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.683622 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:06.683641 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.683697 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.683725 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:06.683744 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.683785 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.683812 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:06.683831 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:06.683869 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.683896 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:06.683915 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:06.683952 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.683982 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:06.684001 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:06.684041 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFs(0=pAsLsXsFs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.684070 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.684096 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.684134 7fb269f70700 client.5557 choose_target_mds starting with req->inode 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFs(0=pAsLsXsFs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.684164 7fb269f70700 client.5557 choose_target_mds 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFs(0=pAsLsXsFs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) is_hash=0 hash=0 2011-11-03 11:14:06.684206 7fb269f70700 client.5557 choose_target_mds from caps on inode 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFs(0=pAsLsXsFs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.684220 7fb269f70700 client.5557 mds is 0 2011-11-03 11:14:06.684233 7fb269f70700 client.5557 send_request rebuilding request 11 for mds.0 2011-11-03 11:14:06.684246 7fb269f70700 client.5557 encode_cap_releases enter (req: 0x417e6230, mds: 0) 2011-11-03 11:14:06.684262 7fb269f70700 client.5557 send_request set sent_stamp to 2011-11-03 11:14:06.684258 2011-11-03 11:14:06.684276 7fb269f70700 client.5557 send_request client_request(unknown.0:11 open #100000000b2) v1 to mds.0 2011-11-03 11:14:06.684295 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5557:11 open #100000000b2) v1 -- ?+0 0x41898290 2011-11-03 11:14:06.684314 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_request(client.5557:11 open #100000000b2) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.684334 7fb269f70700 client.5557 awaiting reply|forward|kick on 0x7fb269f6f010 2011-11-03 11:14:06.684362 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.684388 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 17 0x41898290 client_request(client.5557:11 open #100000000b2) v1 2011-11-03 11:14:06.684410 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 17 0x41898290 2011-11-03 11:14:06.684427 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x41898290 2011-11-03 11:14:06.684455 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.684471 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.684604 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.684638 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.684671 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.684697 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.684719 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.684744 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.684766 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.684789 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=784 src mds.0 front=180 data=0 off 0 2011-11-03 11:14:06.684806 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 180 from dispatch throttler 0/104857600 2011-11-03 11:14:06.684831 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 180 2011-11-03 11:14:06.684869 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.684885 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 180 + 0 + 0 byte message 2011-11-03 11:14:06.684918 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 18 0x418997a0 client_caps(grant ino 100000000b2 6752 seq 7 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v2 2011-11-03 11:14:06.684942 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.684965 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.684995 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.685021 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 18 2011-11-03 11:14:06.685045 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.685079 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x418997a0 2011-11-03 11:14:06.685118 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 18 ==== client_caps(grant ino 100000000b2 6752 seq 7 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v2 ==== 180+0+0 (4014532962 0 0) 0x418997a0 con 0x7fb2709b9a40 2011-11-03 11:14:06.685136 7fb26bb77700 client.5557 mds.0 seq now 4 2011-11-03 11:14:06.685159 7fb26bb77700 client.5557 handle_cap_grant on in 100000000b2 mds.0 seq 7 caps now pAsLsXsFscr was pAsLsXsFs 2011-11-03 11:14:06.685194 7fb26bb77700 client.5557 update_inode_file_bits 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFs(0=pAsLsXsFs) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) pAsLsXsFs mtime 2011-11-03 11:14:05.788394 2011-11-03 11:14:06.685216 7fb26bb77700 client.5557 grant, new caps are Fcr 2011-11-03 11:14:06.685236 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 180 to dispatch throttler 180/104857600 2011-11-03 11:14:06.685254 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x418997a0 2011-11-03 11:14:06.685280 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=26 src mds.0 front=312 data=0 off 0 2011-11-03 11:14:06.685305 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 312 from dispatch throttler 0/104857600 2011-11-03 11:14:06.685332 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.685356 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.685381 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 312 2011-11-03 11:14:06.685420 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.685438 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 312 + 0 + 0 byte message 2011-11-03 11:14:06.685480 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 19 0x41899050 client_reply(???:11 = 0 Success) v1 2011-11-03 11:14:06.685503 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.685527 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.685555 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.685579 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 19 2011-11-03 11:14:06.685604 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x41899050 2011-11-03 11:14:06.685634 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 19 ==== client_reply(???:11 = 0 Success) v1 ==== 312+0+0 (2370865313 0 0) 0x41899050 con 0x7fb2709b9a40 2011-11-03 11:14:06.685652 7fb26bb77700 client.5557 handle_client_reply got a reply. Safe:1 tid 11 2011-11-03 11:14:06.685671 7fb26bb77700 client.5557 insert_trace from 2011-11-03 11:14:06.684258 mds.0 is_target=1 is_dentry=0 2011-11-03 11:14:06.685687 7fb26bb77700 client.5557 features 0xffa 2011-11-03 11:14:06.685704 7fb26bb77700 client.5557 update_snap_trace len 48 2011-11-03 11:14:06.685723 7fb26bb77700 client.5557 update_snap_trace snaprealm(1 nref=9 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.685739 7fb26bb77700 client.5557 hrm is_target=1 is_dentry=0 2011-11-03 11:14:06.685773 7fb26bb77700 client.5557 add_update_inode had 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) caps pAsLsXsFscr 2011-11-03 11:14:06.685810 7fb26bb77700 client.5557 add_update_cap issued pAsLsXsFscr -> pAsLsXsFscr from mds.0 on 100000000b2.head(ref=1 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.685828 7fb26bb77700 client.5557 handle_client_reply signalling caller 0x7fb269f6f010 2011-11-03 11:14:06.685849 7fb26bb77700 client.5557 handle_client_reply awaiting kickback on tid 11 0x7fb26bb76490 2011-11-03 11:14:06.685879 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.685904 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.685931 7fb269f70700 client.5557 sendrecv kickback on tid 11 0x7fb26bb76490 2011-11-03 11:14:06.685953 7fb269f70700 client.5557 lat 0.001691 2011-11-03 11:14:06.685970 7fb269f70700 inode.get on 0x41899360 100000000b2.head now 2 2011-11-03 11:14:06.685983 7fb269f70700 client.5557 _create_fh 100000000b2 mode 1 2011-11-03 11:14:06.685995 7fb269f70700 client.5557 trim_cache size 6 max 16384 2011-11-03 11:14:06.686013 7fb269f70700 client.5557 open exit(#1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0) = 12 2011-11-03 11:14:06.686042 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 312 to dispatch throttler 312/104857600 2011-11-03 11:14:06.686067 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x41899050 2011-11-03 11:14:06.686085 7fb269f70700 client.5557 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:06.686108 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:06.686146 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:06.686161 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.686196 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.686218 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:06.686231 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.686266 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.686288 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:06.686302 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.686335 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.686357 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:06.686371 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:06.686403 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.686424 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:06.686439 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:06.686469 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.686489 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:06.686504 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:06.686537 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=2 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.686577 7fb269f70700 client.5557 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:06.686594 7fb269f70700 client.5557 fill_stat on 100000000b2 snap/devhead mode 0100777 mtime 2011-11-03 11:14:05.788394 ctime 2011-11-03 11:14:05.785578 2011-11-03 11:14:06.686609 7fb269f70700 client.5557 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:06.686628 7fb269f70700 client.5557 open enter(/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0,0) = 2011-11-03 11:14:06.686644 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:06.686666 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:06.686680 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.686714 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.686736 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:06.686750 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.686784 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.686806 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:06.686820 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.686854 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.686875 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:06.686890 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:06.686921 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.686943 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:06.686958 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:06.686988 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.687022 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:06.687037 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:06.687071 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=2 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.687100 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=2 cap_refs={} open={1=2} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted pFscr used - is_delayed=1 2011-11-03 11:14:06.687120 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:06.687152 7fb269f70700 client.5557 send_cap 100000000b2.head(ref=2 cap_refs={} open={1=2} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) mds.0 seq 8 used - want pFscr flush - retain pAsxLsxXsxFsxcrwbl held pAsLsXsFscr revoking - dropping - 2011-11-03 11:14:06.687166 7fb269f70700 client.5557 auth cap, setting max_size = 0 2011-11-03 11:14:06.687192 7fb269f70700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 100000000b2 6752 seq 8 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 -- ?+0 0x418997a0 2011-11-03 11:14:06.687217 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message client_caps(update ino 100000000b2 6752 seq 8 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.687236 7fb269f70700 inode.get on 0x41899360 100000000b2.head now 3 2011-11-03 11:14:06.687253 7fb269f70700 client.5557 _create_fh 100000000b2 mode 1 2011-11-03 11:14:06.687270 7fb269f70700 client.5557 trim_cache size 6 max 16384 2011-11-03 11:14:06.687291 7fb269f70700 client.5557 open exit(#1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0) = 13 2011-11-03 11:14:06.687310 7fb269f70700 client.5557 describe_layout(13) = 0 2011-11-03 11:14:06.687328 7fb269f70700 client.5557 close enter(13) 2011-11-03 11:14:06.687358 7fb269f70700 client.5557 _release_fh 0x41898290 mode 1 on 100000000b2.head(ref=3 cap_refs={} open={1=2} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.687387 7fb269f70700 client.5557 put_inode on 100000000b2.head(ref=3 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.687405 7fb269f70700 inode.put on 0x41899360 100000000b2.head now 2 2011-11-03 11:14:06.687422 7fb269f70700 client.5557 close exit(13) 2011-11-03 11:14:06.687449 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.687482 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 18 0x418997a0 client_caps(update ino 100000000b2 6752 seq 8 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 2011-11-03 11:14:06.687524 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 18 0x418997a0 2011-11-03 11:14:06.687546 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x418997a0 2011-11-03 11:14:06.687574 7fb269f70700 client.5557 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:06.687605 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:06.687634 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:06.687652 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.687694 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:06.687722 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:06.687741 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.687783 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:06.687810 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:06.687829 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.687870 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:06.687897 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:06.687917 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:06.687955 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:06.687981 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:06.688001 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:06.688038 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:06.688079 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:06.688099 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:06.688138 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=2 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.688161 7fb269f70700 client.5557 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:06.688182 7fb269f70700 client.5557 fill_stat on 100000000b2 snap/devhead mode 0100777 mtime 2011-11-03 11:14:05.788394 ctime 2011-11-03 11:14:05.785578 2011-11-03 11:14:06.688202 7fb269f70700 client.5557 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:06.688229 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.688255 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.696980 7fb269f70700 client.5557 get_caps 100000000b2.head(ref=2 cap_refs={} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:06.697036 7fb269f70700 client.5557 lock_fh_pos 0x41898510 2011-11-03 11:14:06.697066 7fb269f70700 client.5557 _read_async 100000000b2.head(ref=2 cap_refs={2048=1} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 0 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 0~4096 2011-11-03 11:14:06.697096 7fb269f70700 client.5557 readahead=0 nr_consec=0 max_byes=0 max_periods=1 2011-11-03 11:14:06.697309 7fb269f70700 -- 192.168.141.130:0/1011511 connect_rank to 192.168.141.127:6800/3516, creating pipe and registering 2011-11-03 11:14:06.697453 7fb269f70700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=-1 pgs=0 cs=0 l=0).register_pipe 2011-11-03 11:14:06.697506 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=-1 pgs=0 cs=0 l=0).writer: state = 1 policy.server=0 2011-11-03 11:14:06.697542 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=-1 pgs=0 cs=0 l=0).connect 0 2011-11-03 11:14:06.697584 7fb269f70700 -- 192.168.141.130:0/1011511 --> 192.168.141.127:6800/3516 -- osd_op(client.5557.0:1 100000000b2.00000000 [read 0~106] 0.37c097a0) v1 -- ?+0 0x41b65dc0 con 0x417e6380 2011-11-03 11:14:06.697613 7fb269f70700 -- 192.168.141.130:0/1011511 submit_message osd_op(client.5557.0:1 100000000b2.00000000 [read 0~106] 0.37c097a0) v1 remote 192.168.141.127:6800/3516 2011-11-03 11:14:06.697646 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=0 cs=0 l=0).connecting to 192.168.141.127:6800/3516 2011-11-03 11:14:06.697981 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=0 cs=0 l=0).connect read peer addr 192.168.141.127:6800/3516 on socket 84 2011-11-03 11:14:06.698006 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=0 cs=0 l=0).connect peer addr for me is 192.168.141.130:47720/0 2011-11-03 11:14:06.698043 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=0 cs=0 l=0).connect sent my addr 192.168.141.130:0/1011511 2011-11-03 11:14:06.698153 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=0 cs=0 l=0).connect.authorizer_len=174 protocol=2 2011-11-03 11:14:06.698177 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=0 cs=0 l=0).connect sending gseq=3 cseq=0 proto=24 2011-11-03 11:14:06.698208 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=0 cs=0 l=0).connect wrote (self +) cseq, waiting for reply 2011-11-03 11:14:06.698418 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=0 cs=0 l=0).connect got reply tag 1 connect_seq 1 global_seq 716 proto 24 flags 1 2011-11-03 11:14:06.698436 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=0 cs=0 l=0).reply.authorizer_len=36 2011-11-03 11:14:06.698496 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).connect success 1, lossy = 1, features 4090 2011-11-03 11:14:06.698515 7fb269e6f700 -- 192.168.141.130:0/1011511 >> :/0 pipe(0x7fb270555340 sd=-1 pgs=0 cs=0 l=0).queue_received queuing pipe 2011-11-03 11:14:06.698538 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).connect starting reader 2011-11-03 11:14:06.698572 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb270555340 dequeued 0 2011-11-03 11:14:06.698599 7fb26bb77700 client.5557 ms_handle_connect on 192.168.141.127:6800/3516 2011-11-03 11:14:06.698642 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.698679 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).writer encoding 1 0x41b65dc0 osd_op(client.5557.0:1 100000000b2.00000000 [read 0~106] 0.37c097a0) v1 2011-11-03 11:14:06.698716 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.698752 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).writer sending 1 0x41b65dc0 2011-11-03 11:14:06.698775 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).write_message 0x41b65dc0 2011-11-03 11:14:06.698810 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.698827 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.699197 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader got ACK 2011-11-03 11:14:06.699222 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.699244 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader got MSG 2011-11-03 11:14:06.699267 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader got envelope type=43 src osd.2 front=111 data=106 off 0 2011-11-03 11:14:06.699284 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader wants 217 from dispatch throttler 0/104857600 2011-11-03 11:14:06.699308 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader got front 111 2011-11-03 11:14:06.699330 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader allocating new rx buffer at offset 0 2011-11-03 11:14:06.699362 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader reading nonblocking into 0x4195c710 len 106 2011-11-03 11:14:06.699396 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).aborted = 0 2011-11-03 11:14:06.699414 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader got 111 + 0 + 106 byte message 2011-11-03 11:14:06.699458 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader got message 1 0x41b65e20 osd_op_reply(1 100000000b2.00000000 [read 0~106] = 0) v2 2011-11-03 11:14:06.699484 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:06.699509 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.699539 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x41899050 dequeued 0x41b65e20 2011-11-03 11:14:06.699572 7fb26bb77700 -- 192.168.141.130:0/1011511 <== osd.2 192.168.141.127:6800/3516 1 ==== osd_op_reply(1 100000000b2.00000000 [read 0~106] = 0) v2 ==== 111+0+106 (4156312425 0 4012969645) 0x41b65e20 con 0x417e6380 2011-11-03 11:14:06.699598 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.699620 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).write_ack 1 2011-11-03 11:14:06.699640 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.699657 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.699679 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 217 to dispatch throttler 217/104857600 2011-11-03 11:14:06.699702 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x41b65e20 2011-11-03 11:14:06.699724 7fb269f70700 client.5557 unlock_fh_pos 0x41898510 2011-11-03 11:14:06.699744 7fb269f70700 client.5557 readahead nr_consec_read 1 for 106 bytes .. last_pos 0 .. offset 0 2011-11-03 11:14:06.699762 7fb269f70700 client.5557 read(12, 0x4195d5a0, 4096, -1) = 106 2011-11-03 11:14:06.699820 7fb269f70700 client.5557 get_caps 100000000b2.head(ref=2 cap_refs={1024=0,2048=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:06.699836 7fb269f70700 client.5557 lock_fh_pos 0x41898510 2011-11-03 11:14:06.699860 7fb269f70700 client.5557 _read_async 100000000b2.head(ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 106~4096 2011-11-03 11:14:06.699873 7fb269f70700 client.5557 unlock_fh_pos 0x41898510 2011-11-03 11:14:06.699886 7fb269f70700 client.5557 readahead nr_consec_read 2 for 106 bytes .. last_pos 106 .. offset 106 2011-11-03 11:14:06.699898 7fb269f70700 client.5557 read(12, 0x4195d5a0, 4096, -1) = 0 2011-11-03 11:14:06.700226 7fb269f70700 client.5557 _fsync(0x41898510, data+metadata) 2011-11-03 11:14:06.700257 7fb269f70700 client.5557 _flush 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.700270 7fb269f70700 client.5557 nothing to flush 2011-11-03 11:14:06.700282 7fb269f70700 client.5557 no metadata needs to commit 2011-11-03 11:14:06.700314 7fb269f70700 client.5557 wait_sync_caps want 0 (last is 0, 0 total flushing) 2011-11-03 11:14:06.700327 7fb269f70700 client.5557 ino 100000000b2 has no uncommitted writes 2011-11-03 11:14:06.700340 7fb269f70700 client.5557 fsync(12, 0) = 0 2011-11-03 11:14:06.700364 7fb269f70700 client.5557 close enter(12) 2011-11-03 11:14:06.700391 7fb269f70700 client.5557 _release_fh 0x41898510 mode 1 on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.700416 7fb269f70700 client.5557 _flush 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.700428 7fb269f70700 client.5557 nothing to flush 2011-11-03 11:14:06.700455 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted Fc used Fc is_delayed=0 2011-11-03 11:14:06.700481 7fb269f70700 client.5557 cap_delay_requeue on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.700503 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:06.700515 7fb269f70700 client.5557 delaying cap release 2011-11-03 11:14:06.700540 7fb269f70700 client.5557 put_inode on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:06.700553 7fb269f70700 inode.put on 0x41899360 100000000b2.head now 1 2011-11-03 11:14:06.700566 7fb269f70700 client.5557 close exit(12) 2011-11-03 11:14:06.718194 7fb269f70700 -- :/0 register_entity client.? 2011-11-03 11:14:06.718247 7fb269f70700 -- :/0 register_entity client.? at :/0 2011-11-03 11:14:06.718284 7fb269f70700 -- :/0 messenger.start 2011-11-03 11:14:06.718346 7fb26946c700 -- :/2011511 reaper_entry start 2011-11-03 11:14:06.718375 7fb26946c700 -- :/2011511 reaper 2011-11-03 11:14:06.718392 7fb26946c700 -- :/2011511 reaper done 2011-11-03 11:14:06.718421 7fb269f70700 -- :/2011511 ready :/2011511 2011-11-03 11:14:06.718751 7fb269f70700 -- :/2011511 connect_rank to 192.168.141.123:6789/0, creating pipe and registering 2011-11-03 11:14:06.718814 7fb269f70700 -- :/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=-1 pgs=0 cs=0 l=0).register_pipe 2011-11-03 11:14:06.718846 7fb269f70700 -- :/2011511 --> 192.168.141.123:6789/0 -- auth(proto 0 30 bytes) v1 -- ?+0 0x41c001d0 con 0x41c05490 2011-11-03 11:14:06.718889 7fb266466700 -- :/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=-1 pgs=0 cs=0 l=0).writer: state = 1 policy.server=0 2011-11-03 11:14:06.718916 7fb266466700 -- :/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=-1 pgs=0 cs=0 l=0).connect 0 2011-11-03 11:14:06.718943 7fb269f70700 -- :/2011511 submit_message auth(proto 0 30 bytes) v1 remote 192.168.141.123:6789/0 2011-11-03 11:14:06.718980 7fb266466700 -- :/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=0 cs=0 l=0).connecting to 192.168.141.123:6789/0 2011-11-03 11:14:06.719401 7fb266466700 -- :/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=0 cs=0 l=0).connect read peer addr 192.168.141.123:6789/0 on socket 83 2011-11-03 11:14:06.719456 7fb266466700 -- :/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=0 cs=0 l=0).connect peer addr for me is 192.168.141.130:34023/0 2011-11-03 11:14:06.719497 7fb266466700 -- 192.168.141.130:0/2011511 learned my addr 192.168.141.130:0/2011511 2011-11-03 11:14:06.719541 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=0 cs=0 l=0).connect sent my addr 192.168.141.130:0/2011511 2011-11-03 11:14:06.719561 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=0 cs=0 l=0).connect sending gseq=1 cseq=0 proto=15 2011-11-03 11:14:06.719603 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=0 cs=0 l=0).connect wrote (self +) cseq, waiting for reply 2011-11-03 11:14:06.719674 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=0 cs=0 l=0).connect got reply tag 1 connect_seq 1 global_seq 380 proto 15 flags 1 2011-11-03 11:14:06.719695 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).connect success 1, lossy = 1, features 4090 2011-11-03 11:14:06.719716 7fb266466700 -- 192.168.141.130:0/2011511 >> :/0 pipe(0x418837f0 sd=-1 pgs=0 cs=0 l=0).queue_received queuing pipe 2011-11-03 11:14:06.719743 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).connect starting reader 2011-11-03 11:14:06.719798 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x418837f0 dequeued 0 2011-11-03 11:14:06.719824 7fb267c69700 client.-1 ms_handle_connect on 192.168.141.123:6789/0 2011-11-03 11:14:06.719855 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.719886 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer encoding 1 0x41c001d0 auth(proto 0 30 bytes) v1 2011-11-03 11:14:06.719923 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.719960 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sending 1 0x41c001d0 2011-11-03 11:14:06.719986 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_message 0x41c001d0 2011-11-03 11:14:06.720022 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.720040 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.720284 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got ACK 2011-11-03 11:14:06.720309 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.720332 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got MSG 2011-11-03 11:14:06.720357 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got envelope type=18 src mon.0 front=33 data=0 off 0 2011-11-03 11:14:06.720375 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader wants 33 from dispatch throttler 0/104857600 2011-11-03 11:14:06.720399 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got front 33 2011-11-03 11:14:06.720422 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).aborted = 0 2011-11-03 11:14:06.720440 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got 33 + 0 + 0 byte message 2011-11-03 11:14:06.720482 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got message 1 0x41c001d0 auth_reply(proto 2 0 Success) v1 2011-11-03 11:14:06.720523 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:06.720548 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.720578 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c05220 dequeued 0x41c001d0 2011-11-03 11:14:06.720609 7fb267c69700 -- 192.168.141.130:0/2011511 <== mon.0 192.168.141.123:6789/0 1 ==== auth_reply(proto 2 0 Success) v1 ==== 33+0+0 (3016517022 0 0) 0x41c001d0 con 0x41c05490 2011-11-03 11:14:06.720636 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.720658 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_ack 1 2011-11-03 11:14:06.720678 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.720694 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.720739 7fb267c69700 -- 192.168.141.130:0/2011511 --> 192.168.141.123:6789/0 -- auth(proto 2 32 bytes) v1 -- ?+0 0x41c001d0 con 0x41c05490 2011-11-03 11:14:06.720761 7fb267c69700 -- 192.168.141.130:0/2011511 submit_message auth(proto 2 32 bytes) v1 remote 192.168.141.123:6789/0 2011-11-03 11:14:06.720782 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 33 to dispatch throttler 33/104857600 2011-11-03 11:14:06.720800 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c001d0 2011-11-03 11:14:06.720826 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.720849 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer encoding 2 0x41c001d0 auth(proto 2 32 bytes) v1 2011-11-03 11:14:06.720872 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sending 2 0x41c001d0 2011-11-03 11:14:06.720890 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_message 0x41c001d0 2011-11-03 11:14:06.720922 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.720939 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.721204 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got ACK 2011-11-03 11:14:06.721227 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.721249 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got MSG 2011-11-03 11:14:06.721272 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got envelope type=18 src mon.0 front=206 data=0 off 0 2011-11-03 11:14:06.721289 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader wants 206 from dispatch throttler 0/104857600 2011-11-03 11:14:06.721312 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got front 206 2011-11-03 11:14:06.721335 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).aborted = 0 2011-11-03 11:14:06.721357 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got 206 + 0 + 0 byte message 2011-11-03 11:14:06.721391 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got message 2 0x41c001d0 auth_reply(proto 2 0 Success) v1 2011-11-03 11:14:06.721415 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:06.721439 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.721466 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c05220 dequeued 0x41c001d0 2011-11-03 11:14:06.721495 7fb267c69700 -- 192.168.141.130:0/2011511 <== mon.0 192.168.141.123:6789/0 2 ==== auth_reply(proto 2 0 Success) v1 ==== 206+0+0 (300337283 0 0) 0x41c001d0 con 0x41c05490 2011-11-03 11:14:06.721522 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.721543 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_ack 2 2011-11-03 11:14:06.721563 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.721582 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.721603 7fb267c69700 -- 192.168.141.130:0/2011511 --> 192.168.141.123:6789/0 -- auth(proto 2 165 bytes) v1 -- ?+0 0x41c001d0 con 0x41c05490 2011-11-03 11:14:06.721624 7fb267c69700 -- 192.168.141.130:0/2011511 submit_message auth(proto 2 165 bytes) v1 remote 192.168.141.123:6789/0 2011-11-03 11:14:06.721643 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 206 to dispatch throttler 206/104857600 2011-11-03 11:14:06.721661 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c001d0 2011-11-03 11:14:06.721687 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.721709 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer encoding 3 0x41c001d0 auth(proto 2 165 bytes) v1 2011-11-03 11:14:06.721734 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sending 3 0x41c001d0 2011-11-03 11:14:06.721752 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_message 0x41c001d0 2011-11-03 11:14:06.721789 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.721806 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.722075 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got ACK 2011-11-03 11:14:06.722099 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.722120 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got MSG 2011-11-03 11:14:06.722143 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got envelope type=18 src mon.0 front=409 data=0 off 0 2011-11-03 11:14:06.722160 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader wants 409 from dispatch throttler 0/104857600 2011-11-03 11:14:06.722183 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got front 409 2011-11-03 11:14:06.722220 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).aborted = 0 2011-11-03 11:14:06.722236 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got 409 + 0 + 0 byte message 2011-11-03 11:14:06.722262 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got message 3 0x4187a1c0 auth_reply(proto 2 0 Success) v1 2011-11-03 11:14:06.722285 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:06.722308 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.722333 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c05220 dequeued 0x4187a1c0 2011-11-03 11:14:06.722362 7fb267c69700 -- 192.168.141.130:0/2011511 <== mon.0 192.168.141.123:6789/0 3 ==== auth_reply(proto 2 0 Success) v1 ==== 409+0+0 (2498041616 0 0) 0x4187a1c0 con 0x41c05490 2011-11-03 11:14:06.722389 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.722410 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_ack 3 2011-11-03 11:14:06.722430 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.722451 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.722480 7fb267c69700 -- 192.168.141.130:0/2011511 --> 192.168.141.123:6789/0 -- mon_subscribe({mdsmap=0+,monmap=0+,osdmap=0}) v1 -- ?+0 0x41c003b0 con 0x41c05490 2011-11-03 11:14:06.722502 7fb267c69700 -- 192.168.141.130:0/2011511 submit_message mon_subscribe({mdsmap=0+,monmap=0+,osdmap=0}) v1 remote 192.168.141.123:6789/0 2011-11-03 11:14:06.722530 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.722562 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer encoding 4 0x41c003b0 mon_subscribe({mdsmap=0+,monmap=0+,osdmap=0}) v1 2011-11-03 11:14:06.722579 7fb269f70700 client.5562 mounted: have osdmap 0 and mdsmap 0 2011-11-03 11:14:06.722614 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sending 4 0x41c003b0 2011-11-03 11:14:06.722641 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_message 0x41c003b0 2011-11-03 11:14:06.722665 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:06.722688 7fb269f70700 client.5562 target mds.0 not active, waiting for new mdsmap 2011-11-03 11:14:06.722716 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.722741 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.722766 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 409 to dispatch throttler 409/104857600 2011-11-03 11:14:06.722784 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x4187a1c0 2011-11-03 11:14:06.722865 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got ACK 2011-11-03 11:14:06.722892 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.722975 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got MSG 2011-11-03 11:14:06.723015 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got envelope type=21 src mon.0 front=535 data=0 off 0 2011-11-03 11:14:06.723032 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader wants 535 from dispatch throttler 0/104857600 2011-11-03 11:14:06.723055 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got front 535 2011-11-03 11:14:06.723077 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).aborted = 0 2011-11-03 11:14:06.723093 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got 535 + 0 + 0 byte message 2011-11-03 11:14:06.723118 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got message 4 0x41c01730 mdsmap(e 14) v1 2011-11-03 11:14:06.723142 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:06.723165 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.723190 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c05220 dequeued 0x41c01730 2011-11-03 11:14:06.723219 7fb267c69700 -- 192.168.141.130:0/2011511 <== mon.0 192.168.141.123:6789/0 4 ==== mdsmap(e 14) v1 ==== 535+0+0 (6102218 0 0) 0x41c01730 con 0x41c05490 2011-11-03 11:14:06.723237 7fb267c69700 client.5562 handle_mds_map epoch 14 2011-11-03 11:14:06.723263 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.723289 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_ack 4 2011-11-03 11:14:06.723313 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.723334 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.723358 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got MSG 2011-11-03 11:14:06.723381 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:06.723405 7fb269f70700 client.5562 opening session to mds.0 2011-11-03 11:14:06.723431 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_open) v1 -- ?+0 0x41c003b0 2011-11-03 11:14:06.723455 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_session(request_open) v1 remote, 192.168.141.123:6802/19591, new pipe. 2011-11-03 11:14:06.723475 7fb269f70700 -- 192.168.141.130:0/2011511 connect_rank to 192.168.141.123:6802/19591, creating pipe and registering 2011-11-03 11:14:06.723500 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 535 to dispatch throttler 535/104857600 2011-11-03 11:14:06.723525 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c01730 2011-11-03 11:14:06.723587 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=-1 pgs=0 cs=0 l=0).writer: state = 1 policy.server=0 2011-11-03 11:14:06.723670 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=-1 pgs=0 cs=0 l=0).connect 0 2011-11-03 11:14:06.723702 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got envelope type=4 src mon.0 front=187 data=0 off 0 2011-11-03 11:14:06.723729 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader wants 187 from dispatch throttler 0/104857600 2011-11-03 11:14:06.723781 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got front 187 2011-11-03 11:14:06.723814 7fb269f70700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=0 cs=0 l=0).register_pipe 2011-11-03 11:14:06.723842 7fb269f70700 client.5562 waiting for session to mds.0 to open 2011-11-03 11:14:06.723870 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).aborted = 0 2011-11-03 11:14:06.723895 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got 187 + 0 + 0 byte message 2011-11-03 11:14:06.723924 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got message 5 0x41c02570 mon_map v1 2011-11-03 11:14:06.723952 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:06.723979 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.724009 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got MSG 2011-11-03 11:14:06.724036 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c05220 dequeued 0x41c02570 2011-11-03 11:14:06.724067 7fb267c69700 -- 192.168.141.130:0/2011511 <== mon.0 192.168.141.123:6789/0 5 ==== mon_map v1 ==== 187+0+0 (2564923774 0 0) 0x41c02570 con 0x41c05490 2011-11-03 11:14:06.724094 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got envelope type=41 src mon.0 front=6122 data=0 off 0 2011-11-03 11:14:06.724120 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader wants 6122 from dispatch throttler 187/104857600 2011-11-03 11:14:06.724150 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.724177 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_ack 5 2011-11-03 11:14:06.724202 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.724223 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.724250 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 187 to dispatch throttler 6309/104857600 2011-11-03 11:14:06.724275 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c02570 2011-11-03 11:14:06.724303 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got front 6122 2011-11-03 11:14:06.724343 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=0 cs=0 l=0).connecting to 192.168.141.123:6802/19591 2011-11-03 11:14:06.724376 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).aborted = 0 2011-11-03 11:14:06.724398 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got 6122 + 0 + 0 byte message 2011-11-03 11:14:06.724437 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got message 6 0x41c01000 osd_map(16..16 src has 1..16) v2 2011-11-03 11:14:06.724462 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:06.724491 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.724536 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c05220 dequeued 0x41c01000 2011-11-03 11:14:06.724570 7fb267c69700 -- 192.168.141.130:0/2011511 <== mon.0 192.168.141.123:6789/0 6 ==== osd_map(16..16 src has 1..16) v2 ==== 6122+0+0 (849204460 0 0) 0x41c01000 con 0x41c05490 2011-11-03 11:14:06.724598 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.724623 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_ack 6 2011-11-03 11:14:06.724642 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.724657 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.724676 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got MSG 2011-11-03 11:14:06.724701 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got envelope type=16 src mon.0 front=20 data=0 off 0 2011-11-03 11:14:06.724719 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader wants 20 from dispatch throttler 6122/104857600 2011-11-03 11:14:06.724748 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got front 20 2011-11-03 11:14:06.724784 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).aborted = 0 2011-11-03 11:14:06.724805 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got 20 + 0 + 0 byte message 2011-11-03 11:14:06.724831 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 6122 to dispatch throttler 6142/104857600 2011-11-03 11:14:06.724855 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c01000 2011-11-03 11:14:06.724882 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader got message 7 0x41c02c70 mon_subscribe_ack(300s) v1 2011-11-03 11:14:06.724908 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:06.724935 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c05220 dequeued 0x41c02c70 2011-11-03 11:14:06.724964 7fb267c69700 -- 192.168.141.130:0/2011511 <== mon.0 192.168.141.123:6789/0 7 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3759263596 0 0) 0x41c02c70 con 0x41c05490 2011-11-03 11:14:06.724988 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 20 to dispatch throttler 20/104857600 2011-11-03 11:14:06.725005 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c02c70 2011-11-03 11:14:06.725032 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=0 cs=0 l=0).connect read peer addr 192.168.141.123:6802/19591 on socket 85 2011-11-03 11:14:06.725056 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=0 cs=0 l=0).connect peer addr for me is 192.168.141.130:54378/0 2011-11-03 11:14:06.725083 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.725107 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_ack 7 2011-11-03 11:14:06.725130 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.725150 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.725193 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=0 cs=0 l=0).connect sent my addr 192.168.141.130:0/2011511 2011-11-03 11:14:06.725226 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.725275 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=0 cs=0 l=0).connect.authorizer_len=174 protocol=2 2011-11-03 11:14:06.725300 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=0 cs=0 l=0).connect sending gseq=2 cseq=0 proto=32 2011-11-03 11:14:06.725329 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=0 cs=0 l=0).connect wrote (self +) cseq, waiting for reply 2011-11-03 11:14:06.725554 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=0 cs=0 l=0).connect got reply tag 1 connect_seq 1 global_seq 1179 proto 32 flags 0 2011-11-03 11:14:06.725572 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=0 cs=0 l=0).reply.authorizer_len=36 2011-11-03 11:14:06.725631 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).connect success 1, lossy = 0, features 4090 2011-11-03 11:14:06.725649 7fb266264700 -- 192.168.141.130:0/2011511 >> :/0 pipe(0x418837f0 sd=-1 pgs=0 cs=0 l=0).queue_received queuing pipe 2011-11-03 11:14:06.725670 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).connect starting reader 2011-11-03 11:14:06.725696 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x418837f0 dequeued 0 2011-11-03 11:14:06.725721 7fb267c69700 client.5562 ms_handle_connect on 192.168.141.123:6802/19591 2011-11-03 11:14:06.725744 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.725774 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 1 0x41c003b0 client_session(request_open) v1 2011-11-03 11:14:06.725799 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 1 0x41c003b0 2011-11-03 11:14:06.725821 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41c003b0 2011-11-03 11:14:06.725857 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.725893 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.725914 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.889953 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.889989 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.922057 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.922086 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.922108 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.922132 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:14:06.922165 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:14:06.922189 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 28 2011-11-03 11:14:06.922212 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.922228 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:14:06.922255 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 1 0x41c003b0 client_session(open) v1 2011-11-03 11:14:06.922280 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.922305 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.922333 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41c003b0 2011-11-03 11:14:06.922362 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 1 ==== client_session(open) v1 ==== 28+0+0 (2719320884 0 0) 0x41c003b0 con 0x41c02110 2011-11-03 11:14:06.922380 7fb267c69700 client.5562 handle_client_session client_session(open) v1 2011-11-03 11:14:06.922399 7fb267c69700 client.5562 renew_caps mds.0 2011-11-03 11:14:06.922427 7fb267c69700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_renewcaps seq 1) v1 -- ?+0 0x41c02d20 2011-11-03 11:14:06.922453 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.922477 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 1 2011-11-03 11:14:06.922504 7fb267c69700 -- 192.168.141.130:0/2011511 submit_message client_session(request_renewcaps seq 1) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.922528 7fb267c69700 client.5562 connect_mds_targets for mds.0 2011-11-03 11:14:06.922558 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 2 0x41c02d20 client_session(request_renewcaps seq 1) v1 2011-11-03 11:14:06.922588 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 2 0x41c02d20 2011-11-03 11:14:06.922611 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41c02d20 2011-11-03 11:14:06.922636 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:14:06.922658 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c003b0 2011-11-03 11:14:06.922687 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.922725 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.922755 7fb269f70700 client.5562 send_request rebuilding request 1 for mds.0 2011-11-03 11:14:06.922785 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.922820 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.922853 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.922892 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:14:06.922914 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:14:06.922943 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 28 2011-11-03 11:14:06.922974 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.922995 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:14:06.923022 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 2 0x41c003b0 client_session(renewcaps seq 1) v1 2011-11-03 11:14:06.923048 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.923077 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.923108 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x4187a470, mds: 0) 2011-11-03 11:14:06.923136 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:06.923130 2011-11-03 11:14:06.923164 7fb269f70700 client.5562 send_request client_request(unknown.0:1 getattr pAsLsXsFs #1) v1 to mds.0 2011-11-03 11:14:06.923190 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:1 getattr pAsLsXsFs #1) v1 -- ?+0 0x418997a0 2011-11-03 11:14:06.923215 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41c003b0 2011-11-03 11:14:06.923244 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (2614397064 0 0) 0x41c003b0 con 0x41c02110 2011-11-03 11:14:06.923271 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.923296 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 2 2011-11-03 11:14:06.923331 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:1 getattr pAsLsXsFs #1) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.923356 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6e5e0 2011-11-03 11:14:06.923393 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 3 0x418997a0 client_request(client.5562:1 getattr pAsLsXsFs #1) v1 2011-11-03 11:14:06.923423 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 3 0x418997a0 2011-11-03 11:14:06.923443 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x418997a0 2011-11-03 11:14:06.923466 7fb267c69700 client.5562 handle_client_session client_session(renewcaps seq 1) v1 2011-11-03 11:14:06.923496 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.923521 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.923545 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:14:06.923578 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c003b0 2011-11-03 11:14:06.923657 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.923688 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.923710 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.923733 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=316 data=0 off 0 2011-11-03 11:14:06.923749 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 316 from dispatch throttler 0/104857600 2011-11-03 11:14:06.923773 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 316 2011-11-03 11:14:06.923795 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.923811 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 316 + 0 + 0 byte message 2011-11-03 11:14:06.923843 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 3 0x41c02d20 client_reply(???:1 = 0 Success) v1 2011-11-03 11:14:06.923866 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.923890 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.923915 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41c02d20 2011-11-03 11:14:06.923944 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 3 ==== client_reply(???:1 = 0 Success) v1 ==== 316+0+0 (3510126754 0 0) 0x41c02d20 con 0x41c02110 2011-11-03 11:14:06.923962 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 1 2011-11-03 11:14:06.923980 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:06.923130 mds.0 is_target=1 is_dentry=0 2011-11-03 11:14:06.923995 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:06.924010 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:06.924031 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=2 c=0 seq=0 parent=0 my_snaps=[] cached_snapc=0=[]) seq 1 > 0 2011-11-03 11:14:06.924049 7fb267c69700 client.5562 flushing caps on snaprealm(1 nref=2 c=0 seq=0 parent=0 my_snaps=[] cached_snapc=0=[]) 2011-11-03 11:14:06.924068 7fb267c69700 client.5562 invalidate_snaprealm_and_children snaprealm(1 nref=2 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=0=[]) 2011-11-03 11:14:06.924085 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=2 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=0=[]) self|parent updated 2011-11-03 11:14:06.924102 7fb267c69700 client.5562 snapc 1=[] 2011-11-03 11:14:06.924119 7fb267c69700 client.5562 hrm is_target=1 is_dentry=0 2011-11-03 11:14:06.924145 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.924169 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 3 2011-11-03 11:14:06.924193 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.924213 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.924249 7fb267c69700 inode.get on 0x41b66000 1.head now 1 2011-11-03 11:14:06.924276 7fb267c69700 client.5562 add_update_inode adding 1.head(ref=1 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41b66000) caps pAsLsXsFs 2011-11-03 11:14:06.924289 7fb267c69700 client.5562 dir hash is 2 2011-11-03 11:14:06.924306 7fb267c69700 client.5562 update_inode_file_bits 1.head(ref=1 cap_refs={} open={} mode=40755 size=0 mtime=0.000000 caps=- 0x41b66000) - mtime 2011-11-01 21:12:55.797267 2011-11-03 11:14:06.924319 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:06.924349 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXsFs from mds.0 on 1.head(ref=1 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) 2011-11-03 11:14:06.924363 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6e5e0 2011-11-03 11:14:06.924380 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 1 0x7fb267c68490 2011-11-03 11:14:06.924417 7fb269f70700 client.5562 sendrecv kickback on tid 1 0x7fb267c68490 2011-11-03 11:14:06.924438 7fb269f70700 client.5562 lat 0.001304 2011-11-03 11:14:06.924456 7fb269f70700 client.5562 root getattr result=0 2011-11-03 11:14:06.924469 7fb269f70700 inode.get on 0x41b66000 1.head now 2 2011-11-03 11:14:06.924482 7fb269f70700 client.5562 _ll_get 0x41b66000 1 -> 1 2011-11-03 11:14:06.924511 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 316 to dispatch throttler 316/104857600 2011-11-03 11:14:06.924529 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c02d20 2011-11-03 11:14:06.924623 7fb269f70700 client.5562 path_walk #1 2011-11-03 11:14:06.924642 7fb269f70700 client.5562 chdir(/) cwd now 1 2011-11-03 11:14:06.924874 7fb269f70700 client.5562 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:06.924898 7fb269f70700 client.5562 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:06.924924 7fb269f70700 client.5562 0 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp 2011-11-03 11:14:06.924945 7fb269f70700 client.5562 _lookup on #1/tmp 2011-11-03 11:14:06.924968 7fb269f70700 client.5562 choose_target_mds starting with req->inode 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) 2011-11-03 11:14:06.924984 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on tmp => 3207142759 2011-11-03 11:14:06.925010 7fb269f70700 client.5562 choose_target_mds 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) is_hash=1 hash=3207142759 2011-11-03 11:14:06.925032 7fb269f70700 client.5562 choose_target_mds from caps on inode 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) 2011-11-03 11:14:06.925045 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:06.925058 7fb269f70700 client.5562 send_request rebuilding request 2 for mds.0 2011-11-03 11:14:06.925071 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x4187a470, mds: 0) 2011-11-03 11:14:06.925087 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:06.925083 2011-11-03 11:14:06.925101 7fb269f70700 client.5562 send_request client_request(unknown.0:2 lookup #1/tmp) v1 to mds.0 2011-11-03 11:14:06.925119 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:2 lookup #1/tmp) v1 -- ?+0 0x41c02d20 2011-11-03 11:14:06.925138 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:2 lookup #1/tmp) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.925168 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6f020 2011-11-03 11:14:06.925211 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.925239 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 4 0x41c02d20 client_request(client.5562:2 lookup #1/tmp) v1 2011-11-03 11:14:06.925260 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 4 0x41c02d20 2011-11-03 11:14:06.925277 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41c02d20 2011-11-03 11:14:06.925306 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.925323 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.925499 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.925525 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.925547 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.925570 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=582 data=0 off 0 2011-11-03 11:14:06.925586 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 582 from dispatch throttler 0/104857600 2011-11-03 11:14:06.925610 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 582 2011-11-03 11:14:06.925633 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.925649 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 582 + 0 + 0 byte message 2011-11-03 11:14:06.925675 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 4 0x41b66690 client_reply(???:2 = 0 Success) v1 2011-11-03 11:14:06.925698 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.925722 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.925750 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41b66690 2011-11-03 11:14:06.925779 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 4 ==== client_reply(???:2 = 0 Success) v1 ==== 582+0+0 (2152890464 0 0) 0x41b66690 con 0x41c02110 2011-11-03 11:14:06.925797 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 2 2011-11-03 11:14:06.925815 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:06.925083 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.925830 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:06.925846 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:06.925864 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=3 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.925880 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.925906 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.925946 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 4 2011-11-03 11:14:06.925969 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.925989 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.926017 7fb267c69700 client.5562 add_update_inode adding 1000000005c.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41b66970) caps pAsLsXsFs 2011-11-03 11:14:06.926035 7fb267c69700 client.5562 dir hash is 2 2011-11-03 11:14:06.926051 7fb267c69700 client.5562 update_inode_file_bits 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x41b66970) - mtime 2011-11-01 21:06:06.505221 2011-11-03 11:14:06.926064 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:06.926092 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXsFs from mds.0 on 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66970) 2011-11-03 11:14:06.926116 7fb267c69700 client.5562 add_update_inode had 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) caps pAsLsXsFs 2011-11-03 11:14:06.926143 7fb267c69700 client.5562 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) 2011-11-03 11:14:06.926156 7fb267c69700 client.5562 got dirfrag map for 1 frag * to mds 0 2011-11-03 11:14:06.926176 7fb267c69700 client.5562 insert_dentry_inode 'tmp' vino 1000000005c.head in dir 1.head dn 0 2011-11-03 11:14:06.926188 7fb267c69700 inode.get on 0x41b66970 1000000005c.head now 1 2011-11-03 11:14:06.926205 7fb267c69700 client.5562 link dir 0x41b66000 'tmp' to inode 0x41b66970 dn 0x41c00420 (new dn) 2011-11-03 11:14:06.926217 7fb267c69700 inode.get on 0x41b66970 1000000005c.head now 2 2011-11-03 11:14:06.926231 7fb267c69700 client.5562 link inode 0x41b66970 parents now 0x41c00420 2011-11-03 11:14:06.926243 7fb267c69700 inode.put on 0x41b66970 1000000005c.head now 1 2011-11-03 11:14:06.926255 7fb267c69700 client.5562 setting dn offset to 2 2011-11-03 11:14:06.926268 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6f020 2011-11-03 11:14:06.926285 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 2 0x7fb267c68490 2011-11-03 11:14:06.926322 7fb269f70700 client.5562 sendrecv kickback on tid 2 0x7fb267c68490 2011-11-03 11:14:06.926343 7fb269f70700 client.5562 lat 0.001257 2011-11-03 11:14:06.926358 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:06.926392 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp = 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.926414 7fb269f70700 client.5562 1 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins 2011-11-03 11:14:06.926432 7fb269f70700 client.5562 _lookup on #1000000005c/hadoop-nwatkins 2011-11-03 11:14:06.926455 7fb269f70700 client.5562 choose_target_mds starting with req->inode 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.926469 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on hadoop-nwatkins => 2416329197 2011-11-03 11:14:06.926491 7fb269f70700 client.5562 choose_target_mds 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) is_hash=1 hash=2416329197 2011-11-03 11:14:06.926527 7fb269f70700 client.5562 choose_target_mds from caps on inode 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.926540 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:06.926552 7fb269f70700 client.5562 send_request rebuilding request 3 for mds.0 2011-11-03 11:14:06.926565 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x41b66440, mds: 0) 2011-11-03 11:14:06.926580 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:06.926576 2011-11-03 11:14:06.926593 7fb269f70700 client.5562 send_request client_request(unknown.0:3 lookup #1000000005c/hadoop-nwatkins) v1 to mds.0 2011-11-03 11:14:06.926612 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:3 lookup #1000000005c/hadoop-nwatkins) v1 -- ?+0 0x41c02d20 2011-11-03 11:14:06.926631 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:3 lookup #1000000005c/hadoop-nwatkins) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.926650 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6f020 2011-11-03 11:14:06.926678 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.926708 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 5 0x41c02d20 client_request(client.5562:3 lookup #1000000005c/hadoop-nwatkins) v1 2011-11-03 11:14:06.926733 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 5 0x41c02d20 2011-11-03 11:14:06.926754 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41c02d20 2011-11-03 11:14:06.926779 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 582 to dispatch throttler 582/104857600 2011-11-03 11:14:06.926802 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41b66690 2011-11-03 11:14:06.926827 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.926849 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.926970 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.926995 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.927017 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.927040 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=594 data=0 off 0 2011-11-03 11:14:06.927057 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 594 from dispatch throttler 0/104857600 2011-11-03 11:14:06.927080 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 594 2011-11-03 11:14:06.927102 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.927118 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 594 + 0 + 0 byte message 2011-11-03 11:14:06.927151 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 5 0x4187a470 client_reply(???:3 = 0 Success) v1 2011-11-03 11:14:06.927183 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.927207 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.927233 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x4187a470 2011-11-03 11:14:06.927261 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 5 ==== client_reply(???:3 = 0 Success) v1 ==== 594+0+0 (298628073 0 0) 0x4187a470 con 0x41c02110 2011-11-03 11:14:06.927279 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 3 2011-11-03 11:14:06.927296 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:06.926576 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.927312 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:06.927328 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:06.927346 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=4 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.927362 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.927388 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.927413 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 5 2011-11-03 11:14:06.927436 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.927456 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.927484 7fb267c69700 client.5562 add_update_inode adding 1000000005d.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41c03660) caps pAsLsXsFs 2011-11-03 11:14:06.927501 7fb267c69700 client.5562 dir hash is 2 2011-11-03 11:14:06.927517 7fb267c69700 client.5562 update_inode_file_bits 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x41c03660) - mtime 2011-11-01 21:06:06.505963 2011-11-03 11:14:06.927530 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:06.927557 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXsFs from mds.0 on 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41c03660) 2011-11-03 11:14:06.927581 7fb267c69700 client.5562 add_update_inode had 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) caps pAsLsXsFs 2011-11-03 11:14:06.927609 7fb267c69700 client.5562 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.927622 7fb267c69700 client.5562 got dirfrag map for 1000000005c frag * to mds -1 2011-11-03 11:14:06.927637 7fb267c69700 client.5562 insert_dentry_inode 'hadoop-nwatkins' vino 1000000005d.head in dir 1000000005c.head dn 0 2011-11-03 11:14:06.927650 7fb267c69700 inode.get on 0x41c03660 1000000005d.head now 1 2011-11-03 11:14:06.927663 7fb267c69700 client.5562 link dir 0x41b66970 'hadoop-nwatkins' to inode 0x41c03660 dn 0x41899a00 (new dn) 2011-11-03 11:14:06.927675 7fb267c69700 inode.get on 0x41c03660 1000000005d.head now 2 2011-11-03 11:14:06.927687 7fb267c69700 client.5562 link inode 0x41c03660 parents now 0x41899a00 2011-11-03 11:14:06.927705 7fb267c69700 inode.put on 0x41c03660 1000000005d.head now 1 2011-11-03 11:14:06.927724 7fb267c69700 client.5562 setting dn offset to 2 2011-11-03 11:14:06.927737 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6f020 2011-11-03 11:14:06.927753 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 3 0x7fb267c68490 2011-11-03 11:14:06.927790 7fb269f70700 client.5562 sendrecv kickback on tid 3 0x7fb267c68490 2011-11-03 11:14:06.927810 7fb269f70700 client.5562 lat 0.001231 2011-11-03 11:14:06.927826 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:06.927859 7fb269f70700 client.5562 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins = 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.927881 7fb269f70700 client.5562 2 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred 2011-11-03 11:14:06.927899 7fb269f70700 client.5562 _lookup on #1000000005d/mapred 2011-11-03 11:14:06.927921 7fb269f70700 client.5562 choose_target_mds starting with req->inode 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.927935 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on mapred => 1878159916 2011-11-03 11:14:06.927957 7fb269f70700 client.5562 choose_target_mds 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) is_hash=1 hash=1878159916 2011-11-03 11:14:06.927978 7fb269f70700 client.5562 choose_target_mds from caps on inode 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.927991 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:06.928003 7fb269f70700 client.5562 send_request rebuilding request 4 for mds.0 2011-11-03 11:14:06.928016 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x41c040b0, mds: 0) 2011-11-03 11:14:06.928031 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:06.928027 2011-11-03 11:14:06.928044 7fb269f70700 client.5562 send_request client_request(unknown.0:4 lookup #1000000005d/mapred) v1 to mds.0 2011-11-03 11:14:06.928063 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:4 lookup #1000000005d/mapred) v1 -- ?+0 0x41c02d20 2011-11-03 11:14:06.928081 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:4 lookup #1000000005d/mapred) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.928099 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6f020 2011-11-03 11:14:06.928126 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.928155 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 6 0x41c02d20 client_request(client.5562:4 lookup #1000000005d/mapred) v1 2011-11-03 11:14:06.928181 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 6 0x41c02d20 2011-11-03 11:14:06.928201 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41c02d20 2011-11-03 11:14:06.928226 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 594 to dispatch throttler 594/104857600 2011-11-03 11:14:06.928248 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x4187a470 2011-11-03 11:14:06.928340 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.928375 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.928451 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.928476 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.928498 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.928521 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=585 data=0 off 0 2011-11-03 11:14:06.928537 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 585 from dispatch throttler 0/104857600 2011-11-03 11:14:06.928561 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 585 2011-11-03 11:14:06.928586 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.928603 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 585 + 0 + 0 byte message 2011-11-03 11:14:06.928629 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 6 0x4187a470 client_reply(???:4 = 0 Success) v1 2011-11-03 11:14:06.928652 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.928676 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.928701 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x4187a470 2011-11-03 11:14:06.928728 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 6 ==== client_reply(???:4 = 0 Success) v1 ==== 585+0+0 (2548567832 0 0) 0x4187a470 con 0x41c02110 2011-11-03 11:14:06.928746 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 4 2011-11-03 11:14:06.928763 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:06.928027 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.928779 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:06.928794 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:06.928812 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=5 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.928828 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.928854 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.928879 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 6 2011-11-03 11:14:06.928902 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.928922 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.928949 7fb267c69700 client.5562 add_update_inode adding 1000000005e.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41c04380) caps pAsLsXs 2011-11-03 11:14:06.928966 7fb267c69700 client.5562 dir hash is 2 2011-11-03 11:14:06.928987 7fb267c69700 client.5562 update_inode_file_bits 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x41c04380) - mtime 2011-11-03 11:13:37.993864 2011-11-03 11:14:06.929007 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:06.929033 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXs from mds.0 on 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) 0x41c04380) 2011-11-03 11:14:06.929057 7fb267c69700 client.5562 add_update_inode had 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) caps pAsLsXsFs 2011-11-03 11:14:06.929085 7fb267c69700 client.5562 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.929097 7fb267c69700 client.5562 got dirfrag map for 1000000005d frag * to mds -1 2011-11-03 11:14:06.929113 7fb267c69700 client.5562 insert_dentry_inode 'mapred' vino 1000000005e.head in dir 1000000005d.head dn 0 2011-11-03 11:14:06.929125 7fb267c69700 inode.get on 0x41c04380 1000000005e.head now 1 2011-11-03 11:14:06.929138 7fb267c69700 client.5562 link dir 0x41c03660 'mapred' to inode 0x41c04380 dn 0x41c047c0 (new dn) 2011-11-03 11:14:06.929150 7fb267c69700 inode.get on 0x41c04380 1000000005e.head now 2 2011-11-03 11:14:06.929162 7fb267c69700 client.5562 link inode 0x41c04380 parents now 0x41c047c0 2011-11-03 11:14:06.929174 7fb267c69700 inode.put on 0x41c04380 1000000005e.head now 1 2011-11-03 11:14:06.929185 7fb267c69700 client.5562 setting dn offset to 2 2011-11-03 11:14:06.929197 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6f020 2011-11-03 11:14:06.929214 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 4 0x7fb267c68490 2011-11-03 11:14:06.929254 7fb269f70700 client.5562 sendrecv kickback on tid 4 0x7fb267c68490 2011-11-03 11:14:06.929273 7fb269f70700 client.5562 lat 0.001243 2011-11-03 11:14:06.929288 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:06.929320 7fb269f70700 client.5562 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred = 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.929342 7fb269f70700 client.5562 3 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging 2011-11-03 11:14:06.929361 7fb269f70700 client.5562 _lookup on #1000000005e/staging 2011-11-03 11:14:06.929383 7fb269f70700 client.5562 choose_target_mds starting with req->inode 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.929397 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on staging => 875688800 2011-11-03 11:14:06.929418 7fb269f70700 client.5562 choose_target_mds 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) is_hash=1 hash=875688800 2011-11-03 11:14:06.929439 7fb269f70700 client.5562 choose_target_mds from caps on inode 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.929451 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:06.929463 7fb269f70700 client.5562 send_request rebuilding request 5 for mds.0 2011-11-03 11:14:06.929476 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x41b66440, mds: 0) 2011-11-03 11:14:06.929491 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:06.929487 2011-11-03 11:14:06.929519 7fb269f70700 client.5562 send_request client_request(unknown.0:5 lookup #1000000005e/staging) v1 to mds.0 2011-11-03 11:14:06.929538 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:5 lookup #1000000005e/staging) v1 -- ?+0 0x41b66db0 2011-11-03 11:14:06.929556 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:5 lookup #1000000005e/staging) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.929574 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6f020 2011-11-03 11:14:06.929602 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.929632 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 7 0x41b66db0 client_request(client.5562:5 lookup #1000000005e/staging) v1 2011-11-03 11:14:06.929657 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 585 to dispatch throttler 585/104857600 2011-11-03 11:14:06.929680 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x4187a470 2011-11-03 11:14:06.929706 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 7 0x41b66db0 2011-11-03 11:14:06.929728 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41b66db0 2011-11-03 11:14:06.929758 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.929775 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.929948 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.929974 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.929995 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.930018 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=586 data=0 off 0 2011-11-03 11:14:06.930035 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 586 from dispatch throttler 0/104857600 2011-11-03 11:14:06.930058 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 586 2011-11-03 11:14:06.930081 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.930097 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 586 + 0 + 0 byte message 2011-11-03 11:14:06.930122 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 7 0x41c040b0 client_reply(???:5 = 0 Success) v1 2011-11-03 11:14:06.930145 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.930169 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.930193 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41c040b0 2011-11-03 11:14:06.930222 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 7 ==== client_reply(???:5 = 0 Success) v1 ==== 586+0+0 (796994118 0 0) 0x41c040b0 con 0x41c02110 2011-11-03 11:14:06.930259 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 5 2011-11-03 11:14:06.930278 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:06.929487 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.930294 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:06.930309 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:06.930327 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=6 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.930343 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.930369 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.930394 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 7 2011-11-03 11:14:06.930417 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.930437 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.930464 7fb267c69700 client.5562 add_update_inode adding 100000008ad.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41873ca0) caps pAsLsXsFs 2011-11-03 11:14:06.930481 7fb267c69700 client.5562 dir hash is 2 2011-11-03 11:14:06.930497 7fb267c69700 client.5562 update_inode_file_bits 100000008ad.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=0.000000 caps=- 0x41873ca0) - mtime 2011-11-01 21:07:33.717434 2011-11-03 11:14:06.930509 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:06.930536 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXsFs from mds.0 on 100000008ad.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41873ca0) 2011-11-03 11:14:06.930558 7fb267c69700 client.5562 add_update_inode had 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) caps pAsLsXs 2011-11-03 11:14:06.930584 7fb267c69700 client.5562 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.930596 7fb267c69700 client.5562 got dirfrag map for 1000000005e frag * to mds -1 2011-11-03 11:14:06.930612 7fb267c69700 client.5562 insert_dentry_inode 'staging' vino 100000008ad.head in dir 1000000005e.head dn 0 2011-11-03 11:14:06.930624 7fb267c69700 inode.get on 0x41873ca0 100000008ad.head now 1 2011-11-03 11:14:06.930637 7fb267c69700 client.5562 link dir 0x41c04380 'staging' to inode 0x41873ca0 dn 0x41c01160 (new dn) 2011-11-03 11:14:06.930649 7fb267c69700 inode.get on 0x41873ca0 100000008ad.head now 2 2011-11-03 11:14:06.930660 7fb267c69700 client.5562 link inode 0x41873ca0 parents now 0x41c01160 2011-11-03 11:14:06.930672 7fb267c69700 inode.put on 0x41873ca0 100000008ad.head now 1 2011-11-03 11:14:06.930683 7fb267c69700 client.5562 setting dn offset to 2 2011-11-03 11:14:06.930696 7fb267c69700 client.5562 got dentry lease on staging dur 30000ms ttl 2011-11-03 11:14:36.929487 2011-11-03 11:14:06.930709 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6f020 2011-11-03 11:14:06.930725 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 5 0x7fb267c68490 2011-11-03 11:14:06.930765 7fb269f70700 client.5562 sendrecv kickback on tid 5 0x7fb267c68490 2011-11-03 11:14:06.930785 7fb269f70700 client.5562 lat 0.001294 2011-11-03 11:14:06.930800 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:06.930833 7fb269f70700 client.5562 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging = 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.930869 7fb269f70700 client.5562 4 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins 2011-11-03 11:14:06.930887 7fb269f70700 client.5562 _lookup on #100000008ad/nwatkins 2011-11-03 11:14:06.930909 7fb269f70700 client.5562 choose_target_mds starting with req->inode 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.930922 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on nwatkins => 2332913853 2011-11-03 11:14:06.930944 7fb269f70700 client.5562 choose_target_mds 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) is_hash=1 hash=2332913853 2011-11-03 11:14:06.930966 7fb269f70700 client.5562 choose_target_mds from caps on inode 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.930978 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:06.930990 7fb269f70700 client.5562 send_request rebuilding request 6 for mds.0 2011-11-03 11:14:06.931003 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x418746f0, mds: 0) 2011-11-03 11:14:06.931019 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:06.931014 2011-11-03 11:14:06.931032 7fb269f70700 client.5562 send_request client_request(unknown.0:6 lookup #100000008ad/nwatkins) v1 to mds.0 2011-11-03 11:14:06.931050 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:6 lookup #100000008ad/nwatkins) v1 -- ?+0 0x418749f0 2011-11-03 11:14:06.931069 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:6 lookup #100000008ad/nwatkins) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.931086 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6f020 2011-11-03 11:14:06.931114 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.931143 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 8 0x418749f0 client_request(client.5562:6 lookup #100000008ad/nwatkins) v1 2011-11-03 11:14:06.931168 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 8 0x418749f0 2011-11-03 11:14:06.931189 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x418749f0 2011-11-03 11:14:06.931213 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 586 to dispatch throttler 586/104857600 2011-11-03 11:14:06.931236 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c040b0 2011-11-03 11:14:06.931262 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.931284 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.931418 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.931443 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.931471 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.931503 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=587 data=0 off 0 2011-11-03 11:14:06.931520 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 587 from dispatch throttler 0/104857600 2011-11-03 11:14:06.931543 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 587 2011-11-03 11:14:06.931566 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.931583 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 587 + 0 + 0 byte message 2011-11-03 11:14:06.931608 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 8 0x41c040b0 client_reply(???:6 = 0 Success) v1 2011-11-03 11:14:06.931631 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.931655 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.931680 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41c040b0 2011-11-03 11:14:06.931710 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 8 ==== client_reply(???:6 = 0 Success) v1 ==== 587+0+0 (2917474703 0 0) 0x41c040b0 con 0x41c02110 2011-11-03 11:14:06.931726 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 6 2011-11-03 11:14:06.931744 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:06.931014 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.931759 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:06.931775 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:06.931793 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=7 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.931809 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.931835 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.931861 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 8 2011-11-03 11:14:06.931883 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.931903 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.931931 7fb267c69700 client.5562 add_update_inode adding 100000008ae.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41874a20) caps pAsLsXsFs 2011-11-03 11:14:06.931948 7fb267c69700 client.5562 dir hash is 2 2011-11-03 11:14:06.931964 7fb267c69700 client.5562 update_inode_file_bits 100000008ae.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=0.000000 caps=- 0x41874a20) - mtime 2011-11-01 21:07:33.718516 2011-11-03 11:14:06.931977 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:06.932003 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXsFs from mds.0 on 100000008ae.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41874a20) 2011-11-03 11:14:06.932027 7fb267c69700 client.5562 add_update_inode had 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) caps pAsLsXsFs 2011-11-03 11:14:06.932069 7fb267c69700 client.5562 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.932082 7fb267c69700 client.5562 got dirfrag map for 100000008ad frag * to mds -1 2011-11-03 11:14:06.932097 7fb267c69700 client.5562 insert_dentry_inode 'nwatkins' vino 100000008ae.head in dir 100000008ad.head dn 0 2011-11-03 11:14:06.932109 7fb267c69700 inode.get on 0x41874a20 100000008ae.head now 1 2011-11-03 11:14:06.932123 7fb267c69700 client.5562 link dir 0x41873ca0 'nwatkins' to inode 0x41874a20 dn 0x41b66440 (new dn) 2011-11-03 11:14:06.932135 7fb267c69700 inode.get on 0x41874a20 100000008ae.head now 2 2011-11-03 11:14:06.932147 7fb267c69700 client.5562 link inode 0x41874a20 parents now 0x41b66440 2011-11-03 11:14:06.932158 7fb267c69700 inode.put on 0x41874a20 100000008ae.head now 1 2011-11-03 11:14:06.932169 7fb267c69700 client.5562 setting dn offset to 2 2011-11-03 11:14:06.932182 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6f020 2011-11-03 11:14:06.932198 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 6 0x7fb267c68490 2011-11-03 11:14:06.932237 7fb269f70700 client.5562 sendrecv kickback on tid 6 0x7fb267c68490 2011-11-03 11:14:06.932257 7fb269f70700 client.5562 lat 0.001240 2011-11-03 11:14:06.932273 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:06.932306 7fb269f70700 client.5562 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins = 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.932329 7fb269f70700 client.5562 5 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging 2011-11-03 11:14:06.932347 7fb269f70700 client.5562 _lookup on #100000008ae/.staging 2011-11-03 11:14:06.932370 7fb269f70700 client.5562 choose_target_mds starting with req->inode 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.932383 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on .staging => 1933209695 2011-11-03 11:14:06.932405 7fb269f70700 client.5562 choose_target_mds 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) is_hash=1 hash=1933209695 2011-11-03 11:14:06.932428 7fb269f70700 client.5562 choose_target_mds from caps on inode 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.932440 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:06.932452 7fb269f70700 client.5562 send_request rebuilding request 7 for mds.0 2011-11-03 11:14:06.932466 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x41875470, mds: 0) 2011-11-03 11:14:06.932481 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:06.932477 2011-11-03 11:14:06.932495 7fb269f70700 client.5562 send_request client_request(unknown.0:7 lookup #100000008ae/.staging) v1 to mds.0 2011-11-03 11:14:06.932514 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:7 lookup #100000008ae/.staging) v1 -- ?+0 0x41875770 2011-11-03 11:14:06.932532 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:7 lookup #100000008ae/.staging) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.932551 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6f020 2011-11-03 11:14:06.932591 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.932621 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 9 0x41875770 client_request(client.5562:7 lookup #100000008ae/.staging) v1 2011-11-03 11:14:06.932646 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 9 0x41875770 2011-11-03 11:14:06.932667 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41875770 2011-11-03 11:14:06.932691 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 587 to dispatch throttler 587/104857600 2011-11-03 11:14:06.932714 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c040b0 2011-11-03 11:14:06.932741 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.932762 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.932904 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.932929 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.932951 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.932974 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=587 data=0 off 0 2011-11-03 11:14:06.932990 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 587 from dispatch throttler 0/104857600 2011-11-03 11:14:06.933014 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 587 2011-11-03 11:14:06.933036 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.933052 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 587 + 0 + 0 byte message 2011-11-03 11:14:06.933077 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 9 0x41c040b0 client_reply(???:7 = 0 Success) v1 2011-11-03 11:14:06.933101 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.933125 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.933150 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41c040b0 2011-11-03 11:14:06.933178 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 9 ==== client_reply(???:7 = 0 Success) v1 ==== 587+0+0 (3184157056 0 0) 0x41c040b0 con 0x41c02110 2011-11-03 11:14:06.933196 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 7 2011-11-03 11:14:06.933213 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:06.932477 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.933228 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:06.933244 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:06.933262 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=8 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.933283 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.933317 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.933343 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 9 2011-11-03 11:14:06.933366 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.933386 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.933415 7fb267c69700 client.5562 add_update_inode adding 100000008af.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41875740) caps pAsLsXs 2011-11-03 11:14:06.933432 7fb267c69700 client.5562 dir hash is 2 2011-11-03 11:14:06.933448 7fb267c69700 client.5562 update_inode_file_bits 100000008af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=0.000000 caps=- 0x41875740) - mtime 2011-11-03 11:14:02.107013 2011-11-03 11:14:06.933460 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:06.933486 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXs from mds.0 on 100000008af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) 0x41875740) 2011-11-03 11:14:06.933510 7fb267c69700 client.5562 add_update_inode had 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) caps pAsLsXsFs 2011-11-03 11:14:06.933537 7fb267c69700 client.5562 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.933550 7fb267c69700 client.5562 got dirfrag map for 100000008ae frag * to mds -1 2011-11-03 11:14:06.933566 7fb267c69700 client.5562 insert_dentry_inode '.staging' vino 100000008af.head in dir 100000008ae.head dn 0 2011-11-03 11:14:06.933578 7fb267c69700 inode.get on 0x41875740 100000008af.head now 1 2011-11-03 11:14:06.933592 7fb267c69700 client.5562 link dir 0x41874a20 '.staging' to inode 0x41875740 dn 0x41b66620 (new dn) 2011-11-03 11:14:06.933604 7fb267c69700 inode.get on 0x41875740 100000008af.head now 2 2011-11-03 11:14:06.933616 7fb267c69700 client.5562 link inode 0x41875740 parents now 0x41b66620 2011-11-03 11:14:06.933628 7fb267c69700 inode.put on 0x41875740 100000008af.head now 1 2011-11-03 11:14:06.933639 7fb267c69700 client.5562 setting dn offset to 2 2011-11-03 11:14:06.933652 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6f020 2011-11-03 11:14:06.933668 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 7 0x7fb267c68490 2011-11-03 11:14:06.933696 7fb269f70700 client.5562 sendrecv kickback on tid 7 0x7fb267c68490 2011-11-03 11:14:06.933716 7fb269f70700 client.5562 lat 0.001236 2011-11-03 11:14:06.933731 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:06.933763 7fb269f70700 client.5562 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging = 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.933785 7fb269f70700 client.5562 6 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 2011-11-03 11:14:06.933805 7fb269f70700 client.5562 _lookup on #100000008af/job_201111031113_0001 2011-11-03 11:14:06.933827 7fb269f70700 client.5562 choose_target_mds starting with req->inode 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.933855 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on job_201111031113_0001 => 3184690367 2011-11-03 11:14:06.933876 7fb269f70700 client.5562 choose_target_mds 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) is_hash=1 hash=3184690367 2011-11-03 11:14:06.933897 7fb269f70700 client.5562 choose_target_mds from caps on inode 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.933909 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:06.933921 7fb269f70700 client.5562 send_request rebuilding request 8 for mds.0 2011-11-03 11:14:06.933935 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x41876190, mds: 0) 2011-11-03 11:14:06.933950 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:06.933946 2011-11-03 11:14:06.933963 7fb269f70700 client.5562 send_request client_request(unknown.0:8 lookup #100000008af/job_201111031113_0001) v1 to mds.0 2011-11-03 11:14:06.933982 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:8 lookup #100000008af/job_201111031113_0001) v1 -- ?+0 0x418764e0 2011-11-03 11:14:06.934000 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:8 lookup #100000008af/job_201111031113_0001) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.934018 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6f020 2011-11-03 11:14:06.934045 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.934074 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 10 0x418764e0 client_request(client.5562:8 lookup #100000008af/job_201111031113_0001) v1 2011-11-03 11:14:06.934099 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 587 to dispatch throttler 587/104857600 2011-11-03 11:14:06.934121 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c040b0 2011-11-03 11:14:06.934147 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 10 0x418764e0 2011-11-03 11:14:06.934169 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x418764e0 2011-11-03 11:14:06.934199 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.934215 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.934403 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.934429 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.934451 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.934474 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=600 data=0 off 0 2011-11-03 11:14:06.934490 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 600 from dispatch throttler 0/104857600 2011-11-03 11:14:06.934514 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 600 2011-11-03 11:14:06.934557 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.934575 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 600 + 0 + 0 byte message 2011-11-03 11:14:06.934600 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 10 0x41c040b0 client_reply(???:8 = 0 Success) v1 2011-11-03 11:14:06.934623 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.934648 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.934673 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41c040b0 2011-11-03 11:14:06.934700 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 10 ==== client_reply(???:8 = 0 Success) v1 ==== 600+0+0 (1492148138 0 0) 0x41c040b0 con 0x41c02110 2011-11-03 11:14:06.934718 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 8 2011-11-03 11:14:06.934736 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:06.933946 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.934752 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:06.934767 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:06.934787 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=9 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.934803 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.934830 7fb267c69700 client.5562 add_update_inode adding 10000000982.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x418764e0) caps pAsLsXs 2011-11-03 11:14:06.934847 7fb267c69700 client.5562 dir hash is 2 2011-11-03 11:14:06.934867 7fb267c69700 client.5562 update_inode_file_bits 10000000982.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=0.000000 caps=- 0x418764e0) - mtime 2011-11-03 11:14:04.597264 2011-11-03 11:14:06.934885 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:06.934915 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXs from mds.0 on 10000000982.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) 0x418764e0) 2011-11-03 11:14:06.934942 7fb267c69700 client.5562 add_update_inode had 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) caps pAsLsXs 2011-11-03 11:14:06.934974 7fb267c69700 client.5562 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.934992 7fb267c69700 client.5562 got dirfrag map for 100000008af frag * to mds -1 2011-11-03 11:14:06.935012 7fb267c69700 client.5562 insert_dentry_inode 'job_201111031113_0001' vino 10000000982.head in dir 100000008af.head dn 0 2011-11-03 11:14:06.935028 7fb267c69700 inode.get on 0x418764e0 10000000982.head now 1 2011-11-03 11:14:06.935045 7fb267c69700 client.5562 link dir 0x41875740 'job_201111031113_0001' to inode 0x418764e0 dn 0x418755c0 (new dn) 2011-11-03 11:14:06.935062 7fb267c69700 inode.get on 0x418764e0 10000000982.head now 2 2011-11-03 11:14:06.935079 7fb267c69700 client.5562 link inode 0x418764e0 parents now 0x418755c0 2011-11-03 11:14:06.935095 7fb267c69700 inode.put on 0x418764e0 10000000982.head now 1 2011-11-03 11:14:06.935110 7fb267c69700 client.5562 setting dn offset to 2 2011-11-03 11:14:06.935128 7fb267c69700 client.5562 got dentry lease on job_201111031113_0001 dur 30000ms ttl 2011-11-03 11:14:36.933946 2011-11-03 11:14:06.935151 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6f020 2011-11-03 11:14:06.935179 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 8 0x7fb267c68490 2011-11-03 11:14:06.935212 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.935240 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 10 2011-11-03 11:14:06.935264 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.935284 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.935312 7fb269f70700 client.5562 sendrecv kickback on tid 8 0x7fb267c68490 2011-11-03 11:14:06.935333 7fb269f70700 client.5562 lat 0.001383 2011-11-03 11:14:06.935348 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:06.935380 7fb269f70700 client.5562 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 = 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.935402 7fb269f70700 client.5562 7 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml 2011-11-03 11:14:06.935422 7fb269f70700 client.5562 _lookup on #10000000982/job.xml 2011-11-03 11:14:06.935445 7fb269f70700 client.5562 choose_target_mds starting with req->inode 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.935459 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on job.xml => 3152429972 2011-11-03 11:14:06.935480 7fb269f70700 client.5562 choose_target_mds 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) is_hash=1 hash=3152429972 2011-11-03 11:14:06.935501 7fb269f70700 client.5562 choose_target_mds from caps on inode 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.935513 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:06.935526 7fb269f70700 client.5562 send_request rebuilding request 9 for mds.0 2011-11-03 11:14:06.935539 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x41876f30, mds: 0) 2011-11-03 11:14:06.935555 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:06.935550 2011-11-03 11:14:06.935569 7fb269f70700 client.5562 send_request client_request(unknown.0:9 lookup #10000000982/job.xml) v1 to mds.0 2011-11-03 11:14:06.935587 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:9 lookup #10000000982/job.xml) v1 -- ?+0 0x418771c0 2011-11-03 11:14:06.935605 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:9 lookup #10000000982/job.xml) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.935625 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6f020 2011-11-03 11:14:06.935657 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.935685 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 11 0x418771c0 client_request(client.5562:9 lookup #10000000982/job.xml) v1 2011-11-03 11:14:06.935711 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 11 0x418771c0 2011-11-03 11:14:06.935746 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x418771c0 2011-11-03 11:14:06.935772 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 600 to dispatch throttler 600/104857600 2011-11-03 11:14:06.935794 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c040b0 2011-11-03 11:14:06.935824 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.935845 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.935987 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:06.936012 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.936034 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:06.936057 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=586 data=0 off 0 2011-11-03 11:14:06.936074 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 586 from dispatch throttler 0/104857600 2011-11-03 11:14:06.936097 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 586 2011-11-03 11:14:06.936120 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:06.936136 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 586 + 0 + 0 byte message 2011-11-03 11:14:06.936161 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 11 0x41c040b0 client_reply(???:9 = 0 Success) v1 2011-11-03 11:14:06.936185 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:06.936209 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:06.936234 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41c040b0 2011-11-03 11:14:06.936261 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 11 ==== client_reply(???:9 = 0 Success) v1 ==== 586+0+0 (743185923 0 0) 0x41c040b0 con 0x41c02110 2011-11-03 11:14:06.936279 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 9 2011-11-03 11:14:06.936296 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:06.935550 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:06.936312 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:06.936327 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:06.936347 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=10 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:06.936363 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:06.936391 7fb267c69700 client.5562 add_update_inode adding 10000000986.head(ref=0 cap_refs={} open={} mode=100000 size=0 mtime=0.000000 caps=- objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] 0x418771c0) caps pAsLsXsFscr 2011-11-03 11:14:06.936413 7fb267c69700 client.5562 update_inode_file_bits 10000000986.head(ref=0 cap_refs={} open={} mode=100644 size=0 mtime=0.000000 caps=- objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] 0x418771c0) - mtime 2011-11-03 11:14:04.597264 2011-11-03 11:14:06.936444 7fb267c69700 client.5562 size 0 -> 17110 2011-11-03 11:14:06.936462 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:06.936495 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXsFscr from mds.0 on 10000000986.head(ref=0 cap_refs={} open={} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] 0x418771c0) 2011-11-03 11:14:06.936522 7fb267c69700 client.5562 add_update_inode had 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) caps pAsLsXs 2011-11-03 11:14:06.936554 7fb267c69700 client.5562 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.936571 7fb267c69700 client.5562 got dirfrag map for 10000000982 frag * to mds -1 2011-11-03 11:14:06.936591 7fb267c69700 client.5562 insert_dentry_inode 'job.xml' vino 10000000986.head in dir 10000000982.head dn 0 2011-11-03 11:14:06.936607 7fb267c69700 inode.get on 0x418771c0 10000000986.head now 1 2011-11-03 11:14:06.936625 7fb267c69700 client.5562 link dir 0x418764e0 'job.xml' to inode 0x418771c0 dn 0x41875680 (new dn) 2011-11-03 11:14:06.936641 7fb267c69700 inode.get on 0x418771c0 10000000986.head now 2 2011-11-03 11:14:06.936658 7fb267c69700 client.5562 link inode 0x418771c0 parents now 0x41875680 2011-11-03 11:14:06.936674 7fb267c69700 inode.put on 0x418771c0 10000000986.head now 1 2011-11-03 11:14:06.936689 7fb267c69700 client.5562 setting dn offset to 2 2011-11-03 11:14:06.936706 7fb267c69700 client.5562 got dentry lease on job.xml dur 30000ms ttl 2011-11-03 11:14:36.935550 2011-11-03 11:14:06.936723 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6f020 2011-11-03 11:14:06.936744 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 9 0x7fb267c68490 2011-11-03 11:14:06.936777 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.936804 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 11 2011-11-03 11:14:06.936828 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.936848 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.936877 7fb269f70700 client.5562 sendrecv kickback on tid 9 0x7fb267c68490 2011-11-03 11:14:06.936897 7fb269f70700 client.5562 lat 0.001344 2011-11-03 11:14:06.936912 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:06.936947 7fb269f70700 client.5562 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.936967 7fb269f70700 client.5562 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:06.936984 7fb269f70700 client.5562 fill_stat on 10000000986 snap/devhead mode 0100644 mtime 2011-11-03 11:14:04.597264 ctime 2011-11-03 11:14:04.597264 2011-11-03 11:14:06.936999 7fb269f70700 client.5562 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:06.937029 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 586 to dispatch throttler 586/104857600 2011-11-03 11:14:06.937051 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41c040b0 2011-11-03 11:14:06.937091 7fb269f70700 client.5562 open enter(/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0,0) = 2011-11-03 11:14:06.937115 7fb269f70700 client.5562 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:06.937138 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp 2011-11-03 11:14:06.937154 7fb269f70700 client.5562 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.937189 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.937211 7fb269f70700 client.5562 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins 2011-11-03 11:14:06.937226 7fb269f70700 client.5562 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.937260 7fb269f70700 client.5562 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.937283 7fb269f70700 client.5562 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred 2011-11-03 11:14:06.937297 7fb269f70700 client.5562 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.937331 7fb269f70700 client.5562 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.937353 7fb269f70700 client.5562 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging 2011-11-03 11:14:06.937368 7fb269f70700 client.5562 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:36.929487 seq 2 2011-11-03 11:14:06.937400 7fb269f70700 client.5562 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.937422 7fb269f70700 client.5562 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins 2011-11-03 11:14:06.937437 7fb269f70700 client.5562 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.937472 7fb269f70700 client.5562 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.937494 7fb269f70700 client.5562 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging 2011-11-03 11:14:06.937509 7fb269f70700 client.5562 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.937549 7fb269f70700 client.5562 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.937584 7fb269f70700 client.5562 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 2011-11-03 11:14:06.937601 7fb269f70700 client.5562 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.933946 seq 3 2011-11-03 11:14:06.937634 7fb269f70700 client.5562 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.937655 7fb269f70700 client.5562 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml 2011-11-03 11:14:06.937670 7fb269f70700 client.5562 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:36.935550 seq 4 2011-11-03 11:14:06.937704 7fb269f70700 client.5562 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.937736 7fb269f70700 client.5562 check_caps on 10000000986.head(ref=1 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) wanted pFscr used - is_delayed=1 2011-11-03 11:14:06.937756 7fb269f70700 client.5562 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:06.937789 7fb269f70700 client.5562 send_cap 10000000986.head(ref=1 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) mds.0 seq 1 used - want pFscr flush - retain pAsxLsxXsxFsxcrwbl held pAsLsXsFscr revoking - dropping - 2011-11-03 11:14:06.937804 7fb269f70700 client.5562 auth cap, setting max_size = 0 2011-11-03 11:14:06.937832 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 10000000986 6761 seq 1 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 -- ?+0 0x41877c10 2011-11-03 11:14:06.937856 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_caps(update ino 10000000986 6761 seq 1 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:06.937875 7fb269f70700 inode.get on 0x418771c0 10000000986.head now 2 2011-11-03 11:14:06.937894 7fb269f70700 client.5562 _create_fh 10000000986 mode 1 2011-11-03 11:14:06.937912 7fb269f70700 client.5562 trim_cache size 8 max 16384 2011-11-03 11:14:06.937935 7fb269f70700 client.5562 open exit(#1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0) = 10 2011-11-03 11:14:06.937954 7fb269f70700 client.5562 describe_layout(10) = 0 2011-11-03 11:14:06.937972 7fb269f70700 client.5562 close enter(10) 2011-11-03 11:14:06.938003 7fb269f70700 client.5562 _release_fh 0x417e62b0 mode 1 on 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.938045 7fb269f70700 client.5562 _flush 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.938062 7fb269f70700 client.5562 nothing to flush 2011-11-03 11:14:06.938092 7fb269f70700 client.5562 check_caps on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) wanted - used - is_delayed=0 2011-11-03 11:14:06.938121 7fb269f70700 client.5562 cap_delay_requeue on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.938150 7fb269f70700 client.5562 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:06.938167 7fb269f70700 client.5562 delaying cap release 2011-11-03 11:14:06.938195 7fb269f70700 client.5562 put_inode on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.938213 7fb269f70700 inode.put on 0x418771c0 10000000986.head now 1 2011-11-03 11:14:06.938231 7fb269f70700 client.5562 close exit(10) 2011-11-03 11:14:06.938262 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.938305 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 12 0x41877c10 client_caps(update ino 10000000986 6761 seq 1 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 2011-11-03 11:14:06.938330 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 12 0x41877c10 2011-11-03 11:14:06.938347 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41877c10 2011-11-03 11:14:06.938377 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:06.938394 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:06.939206 7fb269f70700 client.5562 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:06.939247 7fb269f70700 client.5562 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:06.939275 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp 2011-11-03 11:14:06.939291 7fb269f70700 client.5562 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.939327 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.939350 7fb269f70700 client.5562 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins 2011-11-03 11:14:06.939363 7fb269f70700 client.5562 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.939404 7fb269f70700 client.5562 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.939436 7fb269f70700 client.5562 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred 2011-11-03 11:14:06.939450 7fb269f70700 client.5562 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.939484 7fb269f70700 client.5562 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.939506 7fb269f70700 client.5562 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging 2011-11-03 11:14:06.939521 7fb269f70700 client.5562 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:36.929487 seq 2 2011-11-03 11:14:06.939554 7fb269f70700 client.5562 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.939576 7fb269f70700 client.5562 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins 2011-11-03 11:14:06.939590 7fb269f70700 client.5562 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.939624 7fb269f70700 client.5562 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.939646 7fb269f70700 client.5562 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging 2011-11-03 11:14:06.939660 7fb269f70700 client.5562 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.939694 7fb269f70700 client.5562 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.939715 7fb269f70700 client.5562 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 2011-11-03 11:14:06.939730 7fb269f70700 client.5562 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.933946 seq 3 2011-11-03 11:14:06.939761 7fb269f70700 client.5562 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.939783 7fb269f70700 client.5562 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml 2011-11-03 11:14:06.939810 7fb269f70700 client.5562 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:36.935550 seq 4 2011-11-03 11:14:06.939845 7fb269f70700 client.5562 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.939864 7fb269f70700 client.5562 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:06.939880 7fb269f70700 client.5562 fill_stat on 10000000986 snap/devhead mode 0100644 mtime 2011-11-03 11:14:04.597264 ctime 2011-11-03 11:14:04.597264 2011-11-03 11:14:06.939895 7fb269f70700 client.5562 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:06.939915 7fb269f70700 client.5562 open enter(/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0,0) = 2011-11-03 11:14:06.939932 7fb269f70700 client.5562 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:06.939954 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp 2011-11-03 11:14:06.939968 7fb269f70700 client.5562 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.940002 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.940024 7fb269f70700 client.5562 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins 2011-11-03 11:14:06.940038 7fb269f70700 client.5562 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.940072 7fb269f70700 client.5562 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.940094 7fb269f70700 client.5562 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred 2011-11-03 11:14:06.940108 7fb269f70700 client.5562 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.940142 7fb269f70700 client.5562 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.940163 7fb269f70700 client.5562 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging 2011-11-03 11:14:06.940178 7fb269f70700 client.5562 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:36.929487 seq 2 2011-11-03 11:14:06.940210 7fb269f70700 client.5562 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.940232 7fb269f70700 client.5562 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins 2011-11-03 11:14:06.940258 7fb269f70700 client.5562 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.940293 7fb269f70700 client.5562 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.940315 7fb269f70700 client.5562 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging 2011-11-03 11:14:06.940329 7fb269f70700 client.5562 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.940363 7fb269f70700 client.5562 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.940384 7fb269f70700 client.5562 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 2011-11-03 11:14:06.940399 7fb269f70700 client.5562 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.933946 seq 3 2011-11-03 11:14:06.940430 7fb269f70700 client.5562 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.940451 7fb269f70700 client.5562 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml 2011-11-03 11:14:06.940466 7fb269f70700 client.5562 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:36.935550 seq 4 2011-11-03 11:14:06.940500 7fb269f70700 client.5562 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.940530 7fb269f70700 client.5562 check_caps on 10000000986.head(ref=1 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) wanted pFscr used - is_delayed=1 2011-11-03 11:14:06.940549 7fb269f70700 client.5562 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:06.940563 7fb269f70700 inode.get on 0x418771c0 10000000986.head now 2 2011-11-03 11:14:06.940575 7fb269f70700 client.5562 _create_fh 10000000986 mode 1 2011-11-03 11:14:06.940587 7fb269f70700 client.5562 trim_cache size 8 max 16384 2011-11-03 11:14:06.940604 7fb269f70700 client.5562 open exit(#1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0) = 11 2011-11-03 11:14:06.940618 7fb269f70700 client.5562 describe_layout(11) = 0 2011-11-03 11:14:06.940631 7fb269f70700 client.5562 close enter(11) 2011-11-03 11:14:06.940655 7fb269f70700 client.5562 _release_fh 0x417e62b0 mode 1 on 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.940692 7fb269f70700 client.5562 _flush 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.940704 7fb269f70700 client.5562 nothing to flush 2011-11-03 11:14:06.940728 7fb269f70700 client.5562 check_caps on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) wanted - used - is_delayed=0 2011-11-03 11:14:06.940752 7fb269f70700 client.5562 cap_delay_requeue on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.940773 7fb269f70700 client.5562 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:06.940785 7fb269f70700 client.5562 delaying cap release 2011-11-03 11:14:06.940808 7fb269f70700 client.5562 put_inode on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.940821 7fb269f70700 inode.put on 0x418771c0 10000000986.head now 1 2011-11-03 11:14:06.940833 7fb269f70700 client.5562 close exit(11) 2011-11-03 11:14:06.940878 7fb269f70700 client.5562 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:06.940895 7fb269f70700 client.5562 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:06.940918 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp 2011-11-03 11:14:06.940931 7fb269f70700 client.5562 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.940966 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.940988 7fb269f70700 client.5562 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins 2011-11-03 11:14:06.941002 7fb269f70700 client.5562 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.941036 7fb269f70700 client.5562 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.941058 7fb269f70700 client.5562 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred 2011-11-03 11:14:06.941072 7fb269f70700 client.5562 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.941106 7fb269f70700 client.5562 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.941127 7fb269f70700 client.5562 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging 2011-11-03 11:14:06.941159 7fb269f70700 client.5562 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:36.929487 seq 2 2011-11-03 11:14:06.941192 7fb269f70700 client.5562 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.941213 7fb269f70700 client.5562 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins 2011-11-03 11:14:06.941227 7fb269f70700 client.5562 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.941261 7fb269f70700 client.5562 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.941284 7fb269f70700 client.5562 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging 2011-11-03 11:14:06.941297 7fb269f70700 client.5562 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.941331 7fb269f70700 client.5562 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.941352 7fb269f70700 client.5562 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 2011-11-03 11:14:06.941367 7fb269f70700 client.5562 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.933946 seq 3 2011-11-03 11:14:06.941398 7fb269f70700 client.5562 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.941420 7fb269f70700 client.5562 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml 2011-11-03 11:14:06.941434 7fb269f70700 client.5562 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:36.935550 seq 4 2011-11-03 11:14:06.941468 7fb269f70700 client.5562 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.941486 7fb269f70700 client.5562 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:06.941501 7fb269f70700 client.5562 fill_stat on 10000000986 snap/devhead mode 0100644 mtime 2011-11-03 11:14:04.597264 ctime 2011-11-03 11:14:04.597264 2011-11-03 11:14:06.941516 7fb269f70700 client.5562 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:06.941563 7fb269f70700 client.5562 open enter(/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0,0) = 2011-11-03 11:14:06.941581 7fb269f70700 client.5562 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:06.941622 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp 2011-11-03 11:14:06.941637 7fb269f70700 client.5562 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.941671 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.941693 7fb269f70700 client.5562 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins 2011-11-03 11:14:06.941707 7fb269f70700 client.5562 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.941741 7fb269f70700 client.5562 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.941763 7fb269f70700 client.5562 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred 2011-11-03 11:14:06.941777 7fb269f70700 client.5562 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.941811 7fb269f70700 client.5562 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.941832 7fb269f70700 client.5562 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging 2011-11-03 11:14:06.941847 7fb269f70700 client.5562 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:36.929487 seq 2 2011-11-03 11:14:06.941879 7fb269f70700 client.5562 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.941900 7fb269f70700 client.5562 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins 2011-11-03 11:14:06.941915 7fb269f70700 client.5562 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.941951 7fb269f70700 client.5562 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.941973 7fb269f70700 client.5562 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging 2011-11-03 11:14:06.941986 7fb269f70700 client.5562 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.942050 7fb269f70700 client.5562 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.942088 7fb269f70700 client.5562 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 2011-11-03 11:14:06.942105 7fb269f70700 client.5562 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.933946 seq 3 2011-11-03 11:14:06.942136 7fb269f70700 client.5562 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.942158 7fb269f70700 client.5562 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml 2011-11-03 11:14:06.942173 7fb269f70700 client.5562 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:36.935550 seq 4 2011-11-03 11:14:06.942207 7fb269f70700 client.5562 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.942236 7fb269f70700 client.5562 check_caps on 10000000986.head(ref=1 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) wanted pFscr used - is_delayed=1 2011-11-03 11:14:06.942255 7fb269f70700 client.5562 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:06.942268 7fb269f70700 inode.get on 0x418771c0 10000000986.head now 2 2011-11-03 11:14:06.942280 7fb269f70700 client.5562 _create_fh 10000000986 mode 1 2011-11-03 11:14:06.942293 7fb269f70700 client.5562 trim_cache size 8 max 16384 2011-11-03 11:14:06.942308 7fb269f70700 client.5562 open exit(#1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0) = 12 2011-11-03 11:14:06.942341 7fb269f70700 client.5562 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:06.942358 7fb269f70700 client.5562 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:06.942381 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp 2011-11-03 11:14:06.942395 7fb269f70700 client.5562 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.942429 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.942451 7fb269f70700 client.5562 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins 2011-11-03 11:14:06.942465 7fb269f70700 client.5562 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.942499 7fb269f70700 client.5562 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.942533 7fb269f70700 client.5562 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred 2011-11-03 11:14:06.942547 7fb269f70700 client.5562 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.942581 7fb269f70700 client.5562 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.942603 7fb269f70700 client.5562 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging 2011-11-03 11:14:06.942617 7fb269f70700 client.5562 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:36.929487 seq 2 2011-11-03 11:14:06.942649 7fb269f70700 client.5562 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.942671 7fb269f70700 client.5562 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins 2011-11-03 11:14:06.942685 7fb269f70700 client.5562 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.942719 7fb269f70700 client.5562 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.942741 7fb269f70700 client.5562 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging 2011-11-03 11:14:06.942755 7fb269f70700 client.5562 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.942789 7fb269f70700 client.5562 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.942810 7fb269f70700 client.5562 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 2011-11-03 11:14:06.942825 7fb269f70700 client.5562 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.933946 seq 3 2011-11-03 11:14:06.942856 7fb269f70700 client.5562 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.942878 7fb269f70700 client.5562 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml 2011-11-03 11:14:06.942893 7fb269f70700 client.5562 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:36.935550 seq 4 2011-11-03 11:14:06.942926 7fb269f70700 client.5562 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml = 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.942956 7fb269f70700 client.5562 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:06.942972 7fb269f70700 client.5562 fill_stat on 10000000986 snap/devhead mode 0100644 mtime 2011-11-03 11:14:04.597264 ctime 2011-11-03 11:14:04.597264 2011-11-03 11:14:06.942987 7fb269f70700 client.5562 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:06.943005 7fb269f70700 client.5562 open enter(/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0,0) = 2011-11-03 11:14:06.943021 7fb269f70700 client.5562 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:06.943043 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp 2011-11-03 11:14:06.943057 7fb269f70700 client.5562 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.943091 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.943114 7fb269f70700 client.5562 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins 2011-11-03 11:14:06.943128 7fb269f70700 client.5562 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.943162 7fb269f70700 client.5562 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.943184 7fb269f70700 client.5562 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred 2011-11-03 11:14:06.943198 7fb269f70700 client.5562 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.943231 7fb269f70700 client.5562 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.943253 7fb269f70700 client.5562 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging 2011-11-03 11:14:06.943268 7fb269f70700 client.5562 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:36.929487 seq 2 2011-11-03 11:14:06.943300 7fb269f70700 client.5562 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.943322 7fb269f70700 client.5562 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins 2011-11-03 11:14:06.943336 7fb269f70700 client.5562 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.943370 7fb269f70700 client.5562 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.943405 7fb269f70700 client.5562 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging 2011-11-03 11:14:06.943420 7fb269f70700 client.5562 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.943454 7fb269f70700 client.5562 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.943475 7fb269f70700 client.5562 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 2011-11-03 11:14:06.943490 7fb269f70700 client.5562 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.933946 seq 3 2011-11-03 11:14:06.943521 7fb269f70700 client.5562 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.943542 7fb269f70700 client.5562 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml 2011-11-03 11:14:06.943557 7fb269f70700 client.5562 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:36.935550 seq 4 2011-11-03 11:14:06.943591 7fb269f70700 client.5562 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml = 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.943619 7fb269f70700 client.5562 check_caps on 10000000986.head(ref=2 cap_refs={} open={1=2} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) wanted pFscr used - is_delayed=1 2011-11-03 11:14:06.943639 7fb269f70700 client.5562 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:06.943652 7fb269f70700 inode.get on 0x418771c0 10000000986.head now 3 2011-11-03 11:14:06.943664 7fb269f70700 client.5562 _create_fh 10000000986 mode 1 2011-11-03 11:14:06.943676 7fb269f70700 client.5562 trim_cache size 8 max 16384 2011-11-03 11:14:06.943691 7fb269f70700 client.5562 open exit(#1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0) = 13 2011-11-03 11:14:06.943704 7fb269f70700 client.5562 describe_layout(13) = 0 2011-11-03 11:14:06.943717 7fb269f70700 client.5562 close enter(13) 2011-11-03 11:14:06.943741 7fb269f70700 client.5562 _release_fh 0x41877f80 mode 1 on 10000000986.head(ref=3 cap_refs={} open={1=2} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.943764 7fb269f70700 client.5562 put_inode on 10000000986.head(ref=3 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.943783 7fb269f70700 inode.put on 0x418771c0 10000000986.head now 2 2011-11-03 11:14:06.943803 7fb269f70700 client.5562 close exit(13) 2011-11-03 11:14:06.943837 7fb269f70700 client.5562 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:06.943854 7fb269f70700 client.5562 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:06.943876 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp 2011-11-03 11:14:06.943890 7fb269f70700 client.5562 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.943924 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:06.943946 7fb269f70700 client.5562 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins 2011-11-03 11:14:06.943960 7fb269f70700 client.5562 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.943994 7fb269f70700 client.5562 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:06.944016 7fb269f70700 client.5562 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred 2011-11-03 11:14:06.944030 7fb269f70700 client.5562 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.944063 7fb269f70700 client.5562 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:06.944085 7fb269f70700 client.5562 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging 2011-11-03 11:14:06.944099 7fb269f70700 client.5562 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:36.929487 seq 2 2011-11-03 11:14:06.944131 7fb269f70700 client.5562 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:06.944153 7fb269f70700 client.5562 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins 2011-11-03 11:14:06.944167 7fb269f70700 client.5562 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.944201 7fb269f70700 client.5562 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:06.944223 7fb269f70700 client.5562 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging 2011-11-03 11:14:06.944242 7fb269f70700 client.5562 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:06.944284 7fb269f70700 client.5562 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:06.944306 7fb269f70700 client.5562 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 2011-11-03 11:14:06.944321 7fb269f70700 client.5562 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.933946 seq 3 2011-11-03 11:14:06.944352 7fb269f70700 client.5562 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:06.944373 7fb269f70700 client.5562 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml 2011-11-03 11:14:06.944388 7fb269f70700 client.5562 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:36.935550 seq 4 2011-11-03 11:14:06.944422 7fb269f70700 client.5562 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) job.xml = 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.944439 7fb269f70700 client.5562 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:06.944455 7fb269f70700 client.5562 fill_stat on 10000000986 snap/devhead mode 0100644 mtime 2011-11-03 11:14:04.597264 ctime 2011-11-03 11:14:04.597264 2011-11-03 11:14:06.944470 7fb269f70700 client.5562 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:06.946928 7fb269f70700 client.5562 get_caps 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:06.946949 7fb269f70700 client.5562 lock_fh_pos 0x417e62b0 2011-11-03 11:14:06.946974 7fb269f70700 client.5562 _read_async 10000000986.head(ref=2 cap_refs={2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41875680 0x418771c0) 0~4096 2011-11-03 11:14:06.946990 7fb269f70700 client.5562 readahead=1 nr_consec=0 max_byes=0 max_periods=1 2011-11-03 11:14:06.947063 7fb269f70700 -- 192.168.141.130:0/2011511 connect_rank to 192.168.141.132:6800/3579, creating pipe and registering 2011-11-03 11:14:06.947173 7fb269f70700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=-1 pgs=0 cs=0 l=0).register_pipe 2011-11-03 11:14:06.947218 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=-1 pgs=0 cs=0 l=0).writer: state = 1 policy.server=0 2011-11-03 11:14:06.947252 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=-1 pgs=0 cs=0 l=0).connect 0 2011-11-03 11:14:06.947286 7fb269f70700 -- 192.168.141.130:0/2011511 --> 192.168.141.132:6800/3579 -- osd_op(client.5562.0:1 10000000986.00000000 [read 0~4096] 0.5d1b5ff1) v1 -- ?+0 0x41879010 con 0x41879c30 2011-11-03 11:14:06.947316 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message osd_op(client.5562.0:1 10000000986.00000000 [read 0~4096] 0.5d1b5ff1) v1 remote 192.168.141.132:6800/3579 2011-11-03 11:14:06.947360 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=0 cs=0 l=0).connecting to 192.168.141.132:6800/3579 2011-11-03 11:14:06.947689 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=0 cs=0 l=0).connect read peer addr 192.168.141.132:6800/3579 on socket 88 2011-11-03 11:14:06.947714 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=0 cs=0 l=0).connect peer addr for me is 192.168.141.130:50839/0 2011-11-03 11:14:06.947750 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=0 cs=0 l=0).connect sent my addr 192.168.141.130:0/2011511 2011-11-03 11:14:06.947818 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=0 cs=0 l=0).connect.authorizer_len=174 protocol=2 2011-11-03 11:14:06.947839 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=0 cs=0 l=0).connect sending gseq=3 cseq=0 proto=24 2011-11-03 11:14:06.947869 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=0 cs=0 l=0).connect wrote (self +) cseq, waiting for reply 2011-11-03 11:14:06.948075 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=0 cs=0 l=0).connect got reply tag 1 connect_seq 1 global_seq 578 proto 24 flags 1 2011-11-03 11:14:06.948093 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=0 cs=0 l=0).reply.authorizer_len=36 2011-11-03 11:14:06.948155 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).connect success 1, lossy = 1, features 4090 2011-11-03 11:14:06.948174 7fb266062700 -- 192.168.141.130:0/2011511 >> :/0 pipe(0x418837f0 sd=-1 pgs=0 cs=0 l=0).queue_received queuing pipe 2011-11-03 11:14:06.948196 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).connect starting reader 2011-11-03 11:14:06.948222 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x418837f0 dequeued 0 2011-11-03 11:14:06.948247 7fb267c69700 client.5562 ms_handle_connect on 192.168.141.132:6800/3579 2011-11-03 11:14:06.948288 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.948322 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer encoding 1 0x41879010 osd_op(client.5562.0:1 10000000986.00000000 [read 0~4096] 0.5d1b5ff1) v1 2011-11-03 11:14:06.948359 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.948402 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer sending 1 0x41879010 2011-11-03 11:14:06.948427 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).write_message 0x41879010 2011-11-03 11:14:06.948461 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.948477 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.949016 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got ACK 2011-11-03 11:14:06.949041 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.949063 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got MSG 2011-11-03 11:14:06.949102 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got envelope type=43 src osd.7 front=111 data=4096 off 0 2011-11-03 11:14:06.949119 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader wants 4207 from dispatch throttler 0/104857600 2011-11-03 11:14:06.949143 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got front 111 2011-11-03 11:14:06.949165 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader allocating new rx buffer at offset 0 2011-11-03 11:14:06.949185 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader reading nonblocking into 0x41889000 len 4096 2011-11-03 11:14:06.949215 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).aborted = 0 2011-11-03 11:14:06.949231 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got 111 + 0 + 4096 byte message 2011-11-03 11:14:06.949269 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got message 1 0x41879010 osd_op_reply(1 10000000986.00000000 [read 0~4096] = 0) v2 2011-11-03 11:14:06.949294 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:06.949319 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.949345 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41876f30 dequeued 0x41879010 2011-11-03 11:14:06.949375 7fb267c69700 -- 192.168.141.130:0/2011511 <== osd.7 192.168.141.132:6800/3579 1 ==== osd_op_reply(1 10000000986.00000000 [read 0~4096] = 0) v2 ==== 111+0+4096 (344334805 0 862425587) 0x41879010 con 0x41879c30 2011-11-03 11:14:06.949404 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.949426 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).write_ack 1 2011-11-03 11:14:06.949449 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.949470 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.949508 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 4207 to dispatch throttler 4207/104857600 2011-11-03 11:14:06.949531 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41879010 2011-11-03 11:14:06.949555 7fb269f70700 client.5562 unlock_fh_pos 0x417e62b0 2011-11-03 11:14:06.949575 7fb269f70700 client.5562 readahead nr_consec_read 1 for 4096 bytes .. last_pos 0 .. offset 0 2011-11-03 11:14:06.949589 7fb269f70700 client.5562 read(12, 0x41887230, 4096, -1) = 4096 2011-11-03 11:14:06.949735 7fb269f70700 client.5562 get_caps 10000000986.head(ref=2 cap_refs={1024=0,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:06.949751 7fb269f70700 client.5562 lock_fh_pos 0x417e62b0 2011-11-03 11:14:06.949775 7fb269f70700 client.5562 _read_async 10000000986.head(ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) 4096~4096 2011-11-03 11:14:06.949808 7fb269f70700 client.5562 readahead=1 nr_consec=1 max_byes=0 max_periods=1 2011-11-03 11:14:06.949824 7fb269f70700 client.5562 readahead 1 reads 4096 bytes ... readahead 4096~13014 min 4096 (caller wants 4096~4096) 2011-11-03 11:14:06.949886 7fb269f70700 -- 192.168.141.130:0/2011511 --> 192.168.141.132:6800/3579 -- osd_op(client.5562.0:2 10000000986.00000000 [read 4096~13014] 0.5d1b5ff1) v1 -- ?+0 0x41888240 con 0x41879c30 2011-11-03 11:14:06.949905 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message osd_op(client.5562.0:2 10000000986.00000000 [read 4096~13014] 0.5d1b5ff1) v1 remote 192.168.141.132:6800/3579 2011-11-03 11:14:06.949926 7fb269f70700 client.5562 readahead initiated 2011-11-03 11:14:06.949953 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.949980 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer encoding 2 0x41888240 osd_op(client.5562.0:2 10000000986.00000000 [read 4096~13014] 0.5d1b5ff1) v1 2011-11-03 11:14:06.950002 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer sending 2 0x41888240 2011-11-03 11:14:06.950019 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).write_message 0x41888240 2011-11-03 11:14:06.950051 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.950067 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.950458 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got ACK 2011-11-03 11:14:06.950539 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.950564 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got MSG 2011-11-03 11:14:06.950587 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got envelope type=43 src osd.7 front=111 data=13014 off 4096 2011-11-03 11:14:06.950605 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader wants 13125 from dispatch throttler 0/104857600 2011-11-03 11:14:06.950628 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got front 111 2011-11-03 11:14:06.950650 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader allocating new rx buffer at offset 0 2011-11-03 11:14:06.950670 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader reading nonblocking into 0x4188b000 len 12288 2011-11-03 11:14:06.950705 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader reading nonblocking into 0x41888610 len 726 2011-11-03 11:14:06.950732 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).aborted = 0 2011-11-03 11:14:06.950749 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got 111 + 0 + 13014 byte message 2011-11-03 11:14:06.950787 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader got message 2 0x41888980 osd_op_reply(2 10000000986.00000000 [read 4096~13014] = 0) v2 2011-11-03 11:14:06.950811 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:06.950850 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader reading tag... 2011-11-03 11:14:06.950881 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41876f30 dequeued 0x41888980 2011-11-03 11:14:06.950910 7fb267c69700 -- 192.168.141.130:0/2011511 <== osd.7 192.168.141.132:6800/3579 2 ==== osd_op_reply(2 10000000986.00000000 [read 4096~13014] = 0) v2 ==== 111+0+13014 (3557038930 0 770417814) 0x41888980 con 0x41879c30 2011-11-03 11:14:06.950937 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.950959 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).write_ack 2 2011-11-03 11:14:06.950981 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:06.951002 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer sleeping 2011-11-03 11:14:06.951032 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 13125 to dispatch throttler 13125/104857600 2011-11-03 11:14:06.951055 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41888980 2011-11-03 11:14:06.951077 7fb269f70700 client.5562 unlock_fh_pos 0x417e62b0 2011-11-03 11:14:06.951096 7fb269f70700 client.5562 readahead nr_consec_read 2 for 8192 bytes .. last_pos 4096 .. offset 4096 2011-11-03 11:14:06.951109 7fb269f70700 client.5562 read(12, 0x41887230, 4096, -1) = 4096 2011-11-03 11:14:06.951257 7fb269f70700 client.5562 get_caps 10000000986.head(ref=2 cap_refs={1024=0,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:06.951273 7fb269f70700 client.5562 lock_fh_pos 0x417e62b0 2011-11-03 11:14:06.951297 7fb269f70700 client.5562 _read_async 10000000986.head(ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) 8192~4096 2011-11-03 11:14:06.951311 7fb269f70700 client.5562 readahead=1 nr_consec=2 max_byes=0 max_periods=1 2011-11-03 11:14:06.951324 7fb269f70700 client.5562 readahead 2 reads 8192 bytes ... readahead 8192~8918 min 4096 (caller wants 8192~4096) 2011-11-03 11:14:06.951343 7fb269f70700 client.5562 readahead already have min 2011-11-03 11:14:06.951369 7fb269f70700 client.5562 unlock_fh_pos 0x417e62b0 2011-11-03 11:14:06.951382 7fb269f70700 client.5562 readahead nr_consec_read 3 for 12288 bytes .. last_pos 8192 .. offset 8192 2011-11-03 11:14:06.951396 7fb269f70700 client.5562 read(12, 0x4188e010, 4096, -1) = 4096 2011-11-03 11:14:06.951530 7fb269f70700 client.5562 get_caps 10000000986.head(ref=2 cap_refs={1024=0,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:06.951546 7fb269f70700 client.5562 lock_fh_pos 0x417e62b0 2011-11-03 11:14:06.951571 7fb269f70700 client.5562 _read_async 10000000986.head(ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) 12288~4096 2011-11-03 11:14:06.951585 7fb269f70700 client.5562 readahead=1 nr_consec=3 max_byes=0 max_periods=1 2011-11-03 11:14:06.951598 7fb269f70700 client.5562 readahead 3 reads 12288 bytes ... readahead 12288~4822 min 2411 (caller wants 12288~4096) 2011-11-03 11:14:06.951615 7fb269f70700 client.5562 readahead already have min 2011-11-03 11:14:06.951650 7fb269f70700 client.5562 unlock_fh_pos 0x417e62b0 2011-11-03 11:14:06.951664 7fb269f70700 client.5562 readahead nr_consec_read 4 for 16384 bytes .. last_pos 12288 .. offset 12288 2011-11-03 11:14:06.951677 7fb269f70700 client.5562 read(12, 0x4188e010, 4096, -1) = 4096 2011-11-03 11:14:06.951805 7fb269f70700 client.5562 get_caps 10000000986.head(ref=2 cap_refs={1024=0,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:06.951820 7fb269f70700 client.5562 lock_fh_pos 0x417e62b0 2011-11-03 11:14:06.951845 7fb269f70700 client.5562 _read_async 10000000986.head(ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) 16384~4096 2011-11-03 11:14:06.951858 7fb269f70700 client.5562 readahead=0 nr_consec=4 max_byes=0 max_periods=1 2011-11-03 11:14:06.951878 7fb269f70700 client.5562 unlock_fh_pos 0x417e62b0 2011-11-03 11:14:06.951892 7fb269f70700 client.5562 readahead nr_consec_read 5 for 17110 bytes .. last_pos 16384 .. offset 16384 2011-11-03 11:14:06.951905 7fb269f70700 client.5562 read(12, 0x4188e010, 4096, -1) = 726 2011-11-03 11:14:06.951945 7fb269f70700 client.5562 get_caps 10000000986.head(ref=2 cap_refs={1024=0,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:06.951959 7fb269f70700 client.5562 lock_fh_pos 0x417e62b0 2011-11-03 11:14:06.951983 7fb269f70700 client.5562 _read_async 10000000986.head(ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) 17110~4096 2011-11-03 11:14:06.952016 7fb269f70700 client.5562 unlock_fh_pos 0x417e62b0 2011-11-03 11:14:06.952029 7fb269f70700 client.5562 readahead nr_consec_read 6 for 17110 bytes .. last_pos 17110 .. offset 17110 2011-11-03 11:14:06.952042 7fb269f70700 client.5562 read(12, 0x4188e010, 4096, -1) = 0 2011-11-03 11:14:06.952165 7fb269f70700 client.5562 _fsync(0x417e62b0, data+metadata) 2011-11-03 11:14:06.952193 7fb269f70700 client.5562 _flush 10000000986.head(ref=2 cap_refs={1024=-1,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.952206 7fb269f70700 client.5562 nothing to flush 2011-11-03 11:14:06.952217 7fb269f70700 client.5562 no metadata needs to commit 2011-11-03 11:14:06.952231 7fb269f70700 client.5562 wait_sync_caps want 0 (last is 0, 0 total flushing) 2011-11-03 11:14:06.952244 7fb269f70700 client.5562 ino 10000000986 has no uncommitted writes 2011-11-03 11:14:06.952256 7fb269f70700 client.5562 fsync(12, 0) = 0 2011-11-03 11:14:06.952276 7fb269f70700 client.5562 close enter(12) 2011-11-03 11:14:06.952302 7fb269f70700 client.5562 _release_fh 0x417e62b0 mode 1 on 10000000986.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.952327 7fb269f70700 client.5562 _flush 10000000986.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.952339 7fb269f70700 client.5562 nothing to flush 2011-11-03 11:14:06.952369 7fb269f70700 client.5562 check_caps on 10000000986.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) wanted Fc used Fc is_delayed=0 2011-11-03 11:14:06.952410 7fb269f70700 client.5562 cap_delay_requeue on 10000000986.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.952432 7fb269f70700 client.5562 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:06.952445 7fb269f70700 client.5562 delaying cap release 2011-11-03 11:14:06.952469 7fb269f70700 client.5562 put_inode on 10000000986.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) 2011-11-03 11:14:06.952482 7fb269f70700 inode.put on 0x418771c0 10000000986.head now 1 2011-11-03 11:14:06.952494 7fb269f70700 client.5562 close exit(12) 2011-11-03 11:14:07.010236 7fb269f70700 client.5562 lstat enter (relpath/user/nwatkins/in-dir/_partition.lst mask 341) 2011-11-03 11:14:07.010318 7fb269f70700 client.5562 path_walk #1/user/nwatkins/in-dir/_partition.lst 2011-11-03 11:14:07.010354 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) user 2011-11-03 11:14:07.010379 7fb269f70700 client.5562 _lookup on #1/user 2011-11-03 11:14:07.010405 7fb269f70700 client.5562 choose_target_mds starting with req->inode 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) 2011-11-03 11:14:07.010421 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on user => 2176124347 2011-11-03 11:14:07.010445 7fb269f70700 client.5562 choose_target_mds 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) is_hash=1 hash=2176124347 2011-11-03 11:14:07.010466 7fb269f70700 client.5562 choose_target_mds from caps on inode 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) 2011-11-03 11:14:07.010479 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:07.010493 7fb269f70700 client.5562 send_request rebuilding request 10 for mds.0 2011-11-03 11:14:07.010509 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x41888d70, mds: 0) 2011-11-03 11:14:07.010525 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:07.010520 2011-11-03 11:14:07.010541 7fb269f70700 client.5562 send_request client_request(unknown.0:10 lookup #1/user) v1 to mds.0 2011-11-03 11:14:07.010566 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:10 lookup #1/user) v1 -- ?+0 0x41b68150 2011-11-03 11:14:07.010588 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:10 lookup #1/user) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:07.010617 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6e6b0 2011-11-03 11:14:07.010673 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.010748 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 13 0x41b68150 client_request(client.5562:10 lookup #1/user) v1 2011-11-03 11:14:07.010783 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 13 0x41b68150 2011-11-03 11:14:07.010805 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41b68150 2011-11-03 11:14:07.010897 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.010943 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.011084 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:07.011117 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.011139 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:07.011165 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.011188 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:07.011213 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=583 data=0 off 0 2011-11-03 11:14:07.011231 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 583 from dispatch throttler 0/104857600 2011-11-03 11:14:07.011257 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 583 2011-11-03 11:14:07.011281 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:07.011299 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 583 + 0 + 0 byte message 2011-11-03 11:14:07.011345 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 12 0x41b68150 client_reply(???:10 = 0 Success) v1 2011-11-03 11:14:07.011373 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:07.011400 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.011433 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41b68150 2011-11-03 11:14:07.011468 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 12 ==== client_reply(???:10 = 0 Success) v1 ==== 583+0+0 (1990137769 0 0) 0x41b68150 con 0x41c02110 2011-11-03 11:14:07.011486 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 10 2011-11-03 11:14:07.011504 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:07.010520 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:07.011519 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:07.011535 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:07.011554 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=11 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:07.011570 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:07.011596 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.011622 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 12 2011-11-03 11:14:07.011646 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.011673 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.011714 7fb267c69700 client.5562 add_update_inode adding 10000000ce0.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x4195cbb0) caps pAsLsXs 2011-11-03 11:14:07.011733 7fb267c69700 client.5562 dir hash is 2 2011-11-03 11:14:07.011749 7fb267c69700 client.5562 update_inode_file_bits 10000000ce0.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x4195cbb0) - mtime 2011-11-01 21:12:56.017215 2011-11-03 11:14:07.011762 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:07.011789 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXs from mds.0 on 10000000ce0.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) 0x4195cbb0) 2011-11-03 11:14:07.011813 7fb267c69700 client.5562 add_update_inode had 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) caps pAsLsXsFs 2011-11-03 11:14:07.011840 7fb267c69700 client.5562 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) 2011-11-03 11:14:07.011853 7fb267c69700 client.5562 got dirfrag map for 1 frag * to mds 0 2011-11-03 11:14:07.011867 7fb267c69700 client.5562 insert_dentry_inode 'user' vino 10000000ce0.head in dir 1.head dn 0 2011-11-03 11:14:07.011880 7fb267c69700 inode.get on 0x4195cbb0 10000000ce0.head now 1 2011-11-03 11:14:07.011894 7fb267c69700 client.5562 link dir 0x41b66000 'user' to inode 0x4195cbb0 dn 0x41b68450 (new dn) 2011-11-03 11:14:07.011909 7fb267c69700 inode.get on 0x4195cbb0 10000000ce0.head now 2 2011-11-03 11:14:07.011921 7fb267c69700 client.5562 link inode 0x4195cbb0 parents now 0x41b68450 2011-11-03 11:14:07.011933 7fb267c69700 inode.put on 0x4195cbb0 10000000ce0.head now 1 2011-11-03 11:14:07.011944 7fb267c69700 client.5562 setting dn offset to 3 2011-11-03 11:14:07.011957 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6e6b0 2011-11-03 11:14:07.011975 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 10 0x7fb267c68490 2011-11-03 11:14:07.012025 7fb269f70700 client.5562 sendrecv kickback on tid 10 0x7fb267c68490 2011-11-03 11:14:07.012047 7fb269f70700 client.5562 lat 0.001524 2011-11-03 11:14:07.012068 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:07.012101 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) user = 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) 2011-11-03 11:14:07.012123 7fb269f70700 client.5562 1 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) nwatkins 2011-11-03 11:14:07.012143 7fb269f70700 client.5562 _lookup on #10000000ce0/nwatkins 2011-11-03 11:14:07.012165 7fb269f70700 client.5562 choose_target_mds starting with req->inode 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) 2011-11-03 11:14:07.012179 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on nwatkins => 2332913853 2011-11-03 11:14:07.012200 7fb269f70700 client.5562 choose_target_mds 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) is_hash=1 hash=2332913853 2011-11-03 11:14:07.012221 7fb269f70700 client.5562 choose_target_mds from caps on inode 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) 2011-11-03 11:14:07.012247 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:07.012260 7fb269f70700 client.5562 send_request rebuilding request 11 for mds.0 2011-11-03 11:14:07.012289 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x4188a530, mds: 0) 2011-11-03 11:14:07.012305 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:07.012301 2011-11-03 11:14:07.012319 7fb269f70700 client.5562 send_request client_request(unknown.0:11 lookup #10000000ce0/nwatkins) v1 to mds.0 2011-11-03 11:14:07.012337 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:11 lookup #10000000ce0/nwatkins) v1 -- ?+0 0x4188a800 2011-11-03 11:14:07.012355 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:11 lookup #10000000ce0/nwatkins) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:07.012375 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6e6b0 2011-11-03 11:14:07.012403 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.012433 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 14 0x4188a800 client_request(client.5562:11 lookup #10000000ce0/nwatkins) v1 2011-11-03 11:14:07.012461 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 14 0x4188a800 2011-11-03 11:14:07.012484 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x4188a800 2011-11-03 11:14:07.012513 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 583 to dispatch throttler 583/104857600 2011-11-03 11:14:07.012537 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41b68150 2011-11-03 11:14:07.012563 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.012585 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.012715 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:07.012740 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.012762 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:07.012785 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=587 data=0 off 0 2011-11-03 11:14:07.012802 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 587 from dispatch throttler 0/104857600 2011-11-03 11:14:07.012825 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 587 2011-11-03 11:14:07.012848 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:07.012864 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 587 + 0 + 0 byte message 2011-11-03 11:14:07.012891 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 13 0x41888d70 client_reply(???:11 = 0 Success) v1 2011-11-03 11:14:07.012914 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:07.012938 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.012986 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.013012 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 13 2011-11-03 11:14:07.013040 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41888d70 2011-11-03 11:14:07.013070 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 13 ==== client_reply(???:11 = 0 Success) v1 ==== 587+0+0 (1409146879 0 0) 0x41888d70 con 0x41c02110 2011-11-03 11:14:07.013090 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 11 2011-11-03 11:14:07.013111 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:07.012301 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:07.013129 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:07.013145 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:07.013169 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=12 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:07.013187 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:07.013217 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.013241 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.013276 7fb267c69700 client.5562 add_update_inode adding 10000000ce1.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x4188a800) caps pAsLsXsFs 2011-11-03 11:14:07.013294 7fb267c69700 client.5562 dir hash is 2 2011-11-03 11:14:07.013312 7fb267c69700 client.5562 update_inode_file_bits 10000000ce1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x4188a800) - mtime 2011-11-03 11:14:05.347177 2011-11-03 11:14:07.013327 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:07.013359 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXsFs from mds.0 on 10000000ce1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) 0x4188a800) 2011-11-03 11:14:07.013384 7fb267c69700 client.5562 add_update_inode had 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) caps pAsLsXs 2011-11-03 11:14:07.013411 7fb267c69700 client.5562 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) 2011-11-03 11:14:07.013424 7fb267c69700 client.5562 got dirfrag map for 10000000ce0 frag * to mds -1 2011-11-03 11:14:07.013445 7fb267c69700 client.5562 insert_dentry_inode 'nwatkins' vino 10000000ce1.head in dir 10000000ce0.head dn 0 2011-11-03 11:14:07.013459 7fb267c69700 inode.get on 0x4188a800 10000000ce1.head now 1 2011-11-03 11:14:07.013477 7fb267c69700 client.5562 link dir 0x4195cbb0 'nwatkins' to inode 0x4188a800 dn 0x41879590 (new dn) 2011-11-03 11:14:07.013489 7fb267c69700 inode.get on 0x4188a800 10000000ce1.head now 2 2011-11-03 11:14:07.013502 7fb267c69700 client.5562 link inode 0x4188a800 parents now 0x41879590 2011-11-03 11:14:07.013515 7fb267c69700 inode.put on 0x4188a800 10000000ce1.head now 1 2011-11-03 11:14:07.013526 7fb267c69700 client.5562 setting dn offset to 2 2011-11-03 11:14:07.013541 7fb267c69700 client.5562 got dentry lease on nwatkins dur 30000ms ttl 2011-11-03 11:14:37.012301 2011-11-03 11:14:07.013556 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6e6b0 2011-11-03 11:14:07.013573 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 11 0x7fb267c68490 2011-11-03 11:14:07.013615 7fb269f70700 client.5562 sendrecv kickback on tid 11 0x7fb267c68490 2011-11-03 11:14:07.013646 7fb269f70700 client.5562 lat 0.001342 2011-11-03 11:14:07.013662 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:07.013694 7fb269f70700 client.5562 _lookup 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) nwatkins = 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) 2011-11-03 11:14:07.013717 7fb269f70700 client.5562 2 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) in-dir 2011-11-03 11:14:07.013734 7fb269f70700 client.5562 _lookup on #10000000ce1/in-dir 2011-11-03 11:14:07.013758 7fb269f70700 client.5562 choose_target_mds starting with req->inode 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) 2011-11-03 11:14:07.013770 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on in-dir => 3658178500 2011-11-03 11:14:07.013793 7fb269f70700 client.5562 choose_target_mds 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) is_hash=1 hash=3658178500 2011-11-03 11:14:07.013814 7fb269f70700 client.5562 choose_target_mds from caps on inode 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) 2011-11-03 11:14:07.013827 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:07.013839 7fb269f70700 client.5562 send_request rebuilding request 12 for mds.0 2011-11-03 11:14:07.013853 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x41888d70, mds: 0) 2011-11-03 11:14:07.013869 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:07.013865 2011-11-03 11:14:07.013883 7fb269f70700 client.5562 send_request client_request(unknown.0:12 lookup #10000000ce1/in-dir) v1 to mds.0 2011-11-03 11:14:07.013901 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:12 lookup #10000000ce1/in-dir) v1 -- ?+0 0x41877c10 2011-11-03 11:14:07.013920 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:12 lookup #10000000ce1/in-dir) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:07.013939 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6e6b0 2011-11-03 11:14:07.013972 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.014004 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 15 0x41877c10 client_request(client.5562:12 lookup #10000000ce1/in-dir) v1 2011-11-03 11:14:07.014029 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 587 to dispatch throttler 587/104857600 2011-11-03 11:14:07.014054 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41888d70 2011-11-03 11:14:07.014084 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 15 0x41877c10 2011-11-03 11:14:07.014108 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41877c10 2011-11-03 11:14:07.014137 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.014154 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.014333 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:07.014373 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.014395 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:07.014418 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=585 data=0 off 0 2011-11-03 11:14:07.014435 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 585 from dispatch throttler 0/104857600 2011-11-03 11:14:07.014458 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 585 2011-11-03 11:14:07.014481 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:07.014497 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 585 + 0 + 0 byte message 2011-11-03 11:14:07.014524 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 14 0x41b68150 client_reply(???:12 = 0 Success) v1 2011-11-03 11:14:07.014547 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:07.014571 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.014603 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.014627 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 14 2011-11-03 11:14:07.014651 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41b68150 2011-11-03 11:14:07.014679 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 14 ==== client_reply(???:12 = 0 Success) v1 ==== 585+0+0 (3599318380 0 0) 0x41b68150 con 0x41c02110 2011-11-03 11:14:07.014697 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 12 2011-11-03 11:14:07.014717 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:07.013865 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:07.014733 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:07.014748 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:07.014767 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=13 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:07.014783 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:07.014810 7fb267c69700 client.5562 add_update_inode adding 10000000ce2.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41878610) caps pAsLsXsFs 2011-11-03 11:14:07.014828 7fb267c69700 client.5562 dir hash is 2 2011-11-03 11:14:07.014848 7fb267c69700 client.5562 update_inode_file_bits 10000000ce2.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x41878610) - mtime 2011-11-03 11:14:01.196477 2011-11-03 11:14:07.014866 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:07.014897 7fb267c69700 client.5562 add_update_cap issued - -> pAsLsXsFs from mds.0 on 10000000ce2.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41878610) 2011-11-03 11:14:07.014927 7fb267c69700 client.5562 add_update_inode had 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) caps pAsLsXsFs 2011-11-03 11:14:07.014977 7fb267c69700 client.5562 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) 2011-11-03 11:14:07.014995 7fb267c69700 client.5562 got dirfrag map for 10000000ce1 frag * to mds -1 2011-11-03 11:14:07.015016 7fb267c69700 client.5562 insert_dentry_inode 'in-dir' vino 10000000ce2.head in dir 10000000ce1.head dn 0 2011-11-03 11:14:07.015033 7fb267c69700 inode.get on 0x41878610 10000000ce2.head now 1 2011-11-03 11:14:07.015051 7fb267c69700 client.5562 link dir 0x4188a800 'in-dir' to inode 0x41878610 dn 0x41879240 (new dn) 2011-11-03 11:14:07.015067 7fb267c69700 inode.get on 0x41878610 10000000ce2.head now 2 2011-11-03 11:14:07.015084 7fb267c69700 client.5562 link inode 0x41878610 parents now 0x41879240 2011-11-03 11:14:07.015100 7fb267c69700 inode.put on 0x41878610 10000000ce2.head now 1 2011-11-03 11:14:07.015116 7fb267c69700 client.5562 setting dn offset to 2 2011-11-03 11:14:07.015133 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6e6b0 2011-11-03 11:14:07.015154 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 12 0x7fb267c68490 2011-11-03 11:14:07.015191 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.015217 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.015239 7fb269f70700 client.5562 sendrecv kickback on tid 12 0x7fb267c68490 2011-11-03 11:14:07.015259 7fb269f70700 client.5562 lat 0.001391 2011-11-03 11:14:07.015274 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:07.015307 7fb269f70700 client.5562 _lookup 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) in-dir = 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) 2011-11-03 11:14:07.015330 7fb269f70700 client.5562 3 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) _partition.lst 2011-11-03 11:14:07.015347 7fb269f70700 client.5562 _lookup on #10000000ce2/_partition.lst 2011-11-03 11:14:07.015370 7fb269f70700 client.5562 choose_target_mds starting with req->inode 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) 2011-11-03 11:14:07.015384 7fb269f70700 client.5562 choose_target_mds inode dir hash is 2 on _partition.lst => 1027117801 2011-11-03 11:14:07.015406 7fb269f70700 client.5562 choose_target_mds 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) is_hash=1 hash=1027117801 2011-11-03 11:14:07.015428 7fb269f70700 client.5562 choose_target_mds from caps on inode 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) 2011-11-03 11:14:07.015440 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:07.015452 7fb269f70700 client.5562 send_request rebuilding request 13 for mds.0 2011-11-03 11:14:07.015466 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x4188ac40, mds: 0) 2011-11-03 11:14:07.015481 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:07.015477 2011-11-03 11:14:07.015495 7fb269f70700 client.5562 send_request client_request(unknown.0:13 lookup #10000000ce2/_partition.lst) v1 to mds.0 2011-11-03 11:14:07.015513 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:13 lookup #10000000ce2/_partition.lst) v1 -- ?+0 0x4188a5b0 2011-11-03 11:14:07.015552 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:13 lookup #10000000ce2/_partition.lst) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:07.015570 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6e6b0 2011-11-03 11:14:07.015604 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.015634 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 16 0x4188a5b0 client_request(client.5562:13 lookup #10000000ce2/_partition.lst) v1 2011-11-03 11:14:07.015659 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 585 to dispatch throttler 585/104857600 2011-11-03 11:14:07.015682 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41b68150 2011-11-03 11:14:07.015713 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 16 0x4188a5b0 2011-11-03 11:14:07.015735 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x4188a5b0 2011-11-03 11:14:07.015761 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.015778 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.015960 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:07.015990 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.016013 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:07.016036 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=589 data=0 off 0 2011-11-03 11:14:07.016052 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 589 from dispatch throttler 0/104857600 2011-11-03 11:14:07.016076 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 589 2011-11-03 11:14:07.016098 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:07.016114 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 589 + 0 + 0 byte message 2011-11-03 11:14:07.016141 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 15 0x41888d70 client_reply(???:13 = 0 Success) v1 2011-11-03 11:14:07.016164 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:07.016187 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.016219 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.016242 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 15 2011-11-03 11:14:07.016267 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41888d70 2011-11-03 11:14:07.016295 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 15 ==== client_reply(???:13 = 0 Success) v1 ==== 589+0+0 (3782518073 0 0) 0x41888d70 con 0x41c02110 2011-11-03 11:14:07.016331 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 13 2011-11-03 11:14:07.016351 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:07.015477 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:07.016367 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:07.016382 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:07.016401 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=14 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:07.016418 7fb267c69700 client.5562 hrm is_target=1 is_dentry=1 2011-11-03 11:14:07.016444 7fb267c69700 client.5562 add_update_inode adding 10000000981.head(ref=0 cap_refs={} open={} mode=100000 size=0 mtime=0.000000 caps=- objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x4188e010) caps pLs 2011-11-03 11:14:07.016466 7fb267c69700 client.5562 update_inode_file_bits 10000000981.head(ref=0 cap_refs={} open={} mode=100777 size=0 mtime=0.000000 caps=- objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x4188e010) - mtime 2011-11-03 11:14:01.196477 2011-11-03 11:14:07.016483 7fb267c69700 client.5562 size 0 -> 129 2011-11-03 11:14:07.016500 7fb267c69700 client.5562 add_update_cap first one, opened snaprealm 0x418997a0 2011-11-03 11:14:07.016529 7fb267c69700 client.5562 add_update_cap issued - -> pLs from mds.0 on 10000000981.head(ref=0 cap_refs={} open={} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pLs(0=pLs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x4188e010) 2011-11-03 11:14:07.016558 7fb267c69700 client.5562 add_update_inode had 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) caps pAsLsXsFs 2011-11-03 11:14:07.016591 7fb267c69700 client.5562 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) 2011-11-03 11:14:07.016607 7fb267c69700 client.5562 got dirfrag map for 10000000ce2 frag * to mds -1 2011-11-03 11:14:07.016627 7fb267c69700 client.5562 insert_dentry_inode '_partition.lst' vino 10000000981.head in dir 10000000ce2.head dn 0 2011-11-03 11:14:07.016644 7fb267c69700 inode.get on 0x4188e010 10000000981.head now 1 2011-11-03 11:14:07.016662 7fb267c69700 client.5562 link dir 0x41878610 '_partition.lst' to inode 0x4188e010 dn 0x41879060 (new dn) 2011-11-03 11:14:07.016678 7fb267c69700 inode.get on 0x4188e010 10000000981.head now 2 2011-11-03 11:14:07.016695 7fb267c69700 client.5562 link inode 0x4188e010 parents now 0x41879060 2011-11-03 11:14:07.016712 7fb267c69700 inode.put on 0x4188e010 10000000981.head now 1 2011-11-03 11:14:07.016727 7fb267c69700 client.5562 setting dn offset to 2 2011-11-03 11:14:07.016744 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6e6b0 2011-11-03 11:14:07.016766 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 13 0x7fb267c68490 2011-11-03 11:14:07.016799 7fb269f70700 client.5562 sendrecv kickback on tid 13 0x7fb267c68490 2011-11-03 11:14:07.016824 7fb269f70700 client.5562 lat 0.001344 2011-11-03 11:14:07.016844 7fb269f70700 client.5562 _lookup res is 0 2011-11-03 11:14:07.016883 7fb269f70700 client.5562 _lookup 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) _partition.lst = 10000000981.head(ref=1 cap_refs={} open={} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pLs(0=pLs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.016911 7fb269f70700 client.5562 _getattr mask pAsLsXsFs issued=0 2011-11-03 11:14:07.016942 7fb269f70700 client.5562 choose_target_mds starting with req->inode 10000000981.head(ref=1 cap_refs={} open={} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pLs(0=pLs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.016988 7fb269f70700 client.5562 choose_target_mds 10000000981.head(ref=1 cap_refs={} open={} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pLs(0=pLs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) is_hash=0 hash=0 2011-11-03 11:14:07.017013 7fb269f70700 client.5562 choose_target_mds from caps on inode 10000000981.head(ref=1 cap_refs={} open={} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pLs(0=pLs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.017030 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:07.017047 7fb269f70700 client.5562 send_request rebuilding request 14 for mds.0 2011-11-03 11:14:07.017066 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x41888d70, mds: 0) 2011-11-03 11:14:07.017087 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:07.017082 2011-11-03 11:14:07.017108 7fb269f70700 client.5562 send_request client_request(unknown.0:14 getattr pAsLsXsFs #10000000981) v1 to mds.0 2011-11-03 11:14:07.017135 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:14 getattr pAsLsXsFs #10000000981) v1 -- ?+0 0x4188a5b0 2011-11-03 11:14:07.017166 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.017188 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.017218 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:14 getattr pAsLsXsFs #10000000981) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:07.017241 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6e8d0 2011-11-03 11:14:07.017274 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.017306 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 17 0x4188a5b0 client_request(client.5562:14 getattr pAsLsXsFs #10000000981) v1 2011-11-03 11:14:07.017332 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 589 to dispatch throttler 589/104857600 2011-11-03 11:14:07.017354 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41888d70 2011-11-03 11:14:07.017384 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 17 0x4188a5b0 2011-11-03 11:14:07.017406 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x4188a5b0 2011-11-03 11:14:07.017433 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.017449 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.026771 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:07.026804 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.026825 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:07.026848 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=316 data=0 off 0 2011-11-03 11:14:07.026871 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 316 from dispatch throttler 0/104857600 2011-11-03 11:14:07.026954 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 316 2011-11-03 11:14:07.026978 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:07.026994 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 316 + 0 + 0 byte message 2011-11-03 11:14:07.027021 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 16 0x41b68150 client_reply(???:14 = 0 Success) v1 2011-11-03 11:14:07.027045 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:07.027068 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.027096 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.027120 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 16 2011-11-03 11:14:07.027145 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41b68150 2011-11-03 11:14:07.027173 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 16 ==== client_reply(???:14 = 0 Success) v1 ==== 316+0+0 (3207366239 0 0) 0x41b68150 con 0x41c02110 2011-11-03 11:14:07.027190 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 14 2011-11-03 11:14:07.027208 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:07.017082 mds.0 is_target=1 is_dentry=0 2011-11-03 11:14:07.027223 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:07.027238 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:07.027256 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=15 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:07.027271 7fb267c69700 client.5562 hrm is_target=1 is_dentry=0 2011-11-03 11:14:07.027302 7fb267c69700 client.5562 add_update_inode had 10000000981.head(ref=1 cap_refs={} open={} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pLs(0=pLs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) caps pAsLsXs 2011-11-03 11:14:07.027334 7fb267c69700 client.5562 add_update_cap issued pLs -> pAsLsXs from mds.0 on 10000000981.head(ref=1 cap_refs={} open={} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXs(0=pAsLsXs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.027352 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6e8d0 2011-11-03 11:14:07.027371 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 14 0x7fb267c68490 2011-11-03 11:14:07.027398 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.027422 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.027444 7fb269f70700 client.5562 sendrecv kickback on tid 14 0x7fb267c68490 2011-11-03 11:14:07.027465 7fb269f70700 client.5562 lat 0.010380 2011-11-03 11:14:07.027480 7fb269f70700 client.5562 _getattr result=0 2011-11-03 11:14:07.027497 7fb269f70700 client.5562 fill_stat on 10000000981 snap/devhead mode 0100777 mtime 2011-11-03 11:14:01.196477 ctime 2011-11-03 11:14:01.196477 2011-11-03 11:14:07.027512 7fb269f70700 client.5562 lstat exit (relpath/user/nwatkins/in-dir/_partition.lst mask 341) 2011-11-03 11:14:07.027554 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 316 to dispatch throttler 316/104857600 2011-11-03 11:14:07.027576 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41b68150 2011-11-03 11:14:07.027599 7fb269f70700 client.5562 open enter(/user/nwatkins/in-dir/_partition.lst, 0,0) = 2011-11-03 11:14:07.027621 7fb269f70700 client.5562 path_walk #1/user/nwatkins/in-dir/_partition.lst 2011-11-03 11:14:07.027645 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) user 2011-11-03 11:14:07.027660 7fb269f70700 client.5562 _lookup have dn user mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:07.027695 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) user = 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) 2011-11-03 11:14:07.027717 7fb269f70700 client.5562 1 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) nwatkins 2011-11-03 11:14:07.027732 7fb269f70700 client.5562 _lookup have dn nwatkins mds.0 ttl 2011-11-03 11:14:37.012301 seq 5 2011-11-03 11:14:07.027764 7fb269f70700 client.5562 _lookup 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) nwatkins = 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) 2011-11-03 11:14:07.027787 7fb269f70700 client.5562 2 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) in-dir 2011-11-03 11:14:07.027801 7fb269f70700 client.5562 _lookup have dn in-dir mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:07.027835 7fb269f70700 client.5562 _lookup 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) in-dir = 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) 2011-11-03 11:14:07.027857 7fb269f70700 client.5562 3 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) _partition.lst 2011-11-03 11:14:07.027871 7fb269f70700 client.5562 _lookup have dn _partition.lst mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:07.027906 7fb269f70700 client.5562 _lookup 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) _partition.lst = 10000000981.head(ref=1 cap_refs={} open={} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXs(0=pAsLsXs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.027939 7fb269f70700 client.5562 choose_target_mds starting with req->inode 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXs(0=pAsLsXs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.027963 7fb269f70700 client.5562 choose_target_mds 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXs(0=pAsLsXs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) is_hash=0 hash=0 2011-11-03 11:14:07.027985 7fb269f70700 client.5562 choose_target_mds from caps on inode 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXs(0=pAsLsXs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.028010 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:07.028023 7fb269f70700 client.5562 send_request rebuilding request 15 for mds.0 2011-11-03 11:14:07.028036 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x41888d70, mds: 0) 2011-11-03 11:14:07.028051 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:07.028047 2011-11-03 11:14:07.028064 7fb269f70700 client.5562 send_request client_request(unknown.0:15 open #10000000981) v1 to mds.0 2011-11-03 11:14:07.028083 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:15 open #10000000981) v1 -- ?+0 0x4188a5b0 2011-11-03 11:14:07.028101 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:15 open #10000000981) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:07.028121 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6e8e0 2011-11-03 11:14:07.028147 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.028172 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 18 0x4188a5b0 client_request(client.5562:15 open #10000000981) v1 2011-11-03 11:14:07.028194 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 18 0x4188a5b0 2011-11-03 11:14:07.028212 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x4188a5b0 2011-11-03 11:14:07.028238 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.028255 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.028403 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:07.028434 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.028467 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:07.028490 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=784 src mds.0 front=180 data=0 off 0 2011-11-03 11:14:07.028526 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 180 from dispatch throttler 0/104857600 2011-11-03 11:14:07.028550 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 180 2011-11-03 11:14:07.028572 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:07.028589 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 180 + 0 + 0 byte message 2011-11-03 11:14:07.028627 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 17 0x4188ea60 client_caps(grant ino 10000000981 6765 seq 3 caps=pAsLsXsFcb dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v2 2011-11-03 11:14:07.028650 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:07.028672 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.028707 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.028742 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 17 2011-11-03 11:14:07.028768 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:07.028800 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x4188ea60 2011-11-03 11:14:07.028840 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 17 ==== client_caps(grant ino 10000000981 6765 seq 3 caps=pAsLsXsFcb dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v2 ==== 180+0+0 (187770534 0 0) 0x4188ea60 con 0x41c02110 2011-11-03 11:14:07.028859 7fb267c69700 client.5562 mds.0 seq now 1 2011-11-03 11:14:07.028881 7fb267c69700 client.5562 handle_cap_grant on in 10000000981 mds.0 seq 3 caps now pAsLsXsFcb was pAsLsXs 2011-11-03 11:14:07.028916 7fb267c69700 client.5562 update_inode_file_bits 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXs(0=pAsLsXs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) pAsLsXs mtime 2011-11-03 11:14:01.196477 2011-11-03 11:14:07.028937 7fb267c69700 client.5562 grant, new caps are Fcb 2011-11-03 11:14:07.028957 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 180 to dispatch throttler 180/104857600 2011-11-03 11:14:07.028974 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x4188ea60 2011-11-03 11:14:07.029000 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=312 data=0 off 0 2011-11-03 11:14:07.029024 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 312 from dispatch throttler 0/104857600 2011-11-03 11:14:07.029052 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.029075 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.029101 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 312 2011-11-03 11:14:07.029141 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:07.029158 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 312 + 0 + 0 byte message 2011-11-03 11:14:07.029185 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 18 0x41b68150 client_reply(???:15 = 0 Success) v1 2011-11-03 11:14:07.029208 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:07.029231 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.029259 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.029282 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 18 2011-11-03 11:14:07.029307 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41b68150 2011-11-03 11:14:07.029336 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 18 ==== client_reply(???:15 = 0 Success) v1 ==== 312+0+0 (2006886515 0 0) 0x41b68150 con 0x41c02110 2011-11-03 11:14:07.029366 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 15 2011-11-03 11:14:07.029384 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:07.028047 mds.0 is_target=1 is_dentry=0 2011-11-03 11:14:07.029399 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:07.029415 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:07.029433 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=15 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:07.029448 7fb267c69700 client.5562 hrm is_target=1 is_dentry=0 2011-11-03 11:14:07.029481 7fb267c69700 client.5562 add_update_inode had 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) caps pAsLsXsFcb 2011-11-03 11:14:07.029514 7fb267c69700 client.5562 update_inode_file_bits 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) pAsLsXsFcb mtime 2011-11-03 11:14:01.196477 2011-11-03 11:14:07.029549 7fb267c69700 client.5562 add_update_cap issued pAsLsXsFcb -> pAsLsXsFcb from mds.0 on 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.029567 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6e8e0 2011-11-03 11:14:07.029586 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 15 0x7fb267c68490 2011-11-03 11:14:07.029613 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.029638 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.029660 7fb269f70700 client.5562 sendrecv kickback on tid 15 0x7fb267c68490 2011-11-03 11:14:07.029681 7fb269f70700 client.5562 lat 0.001630 2011-11-03 11:14:07.029698 7fb269f70700 inode.get on 0x4188e010 10000000981.head now 2 2011-11-03 11:14:07.029710 7fb269f70700 client.5562 _create_fh 10000000981 mode 1 2011-11-03 11:14:07.029723 7fb269f70700 client.5562 trim_cache size 12 max 16384 2011-11-03 11:14:07.029741 7fb269f70700 client.5562 open exit(#1/user/nwatkins/in-dir/_partition.lst, 0) = 14 2011-11-03 11:14:07.029761 7fb269f70700 client.5562 describe_layout(14) = 0 2011-11-03 11:14:07.029779 7fb269f70700 client.5562 close enter(14) 2011-11-03 11:14:07.029810 7fb269f70700 client.5562 _release_fh 0x418885b0 mode 1 on 10000000981.head(ref=2 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.029840 7fb269f70700 client.5562 _flush 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.029857 7fb269f70700 client.5562 nothing to flush 2011-11-03 11:14:07.029887 7fb269f70700 client.5562 check_caps on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) wanted - used - is_delayed=0 2011-11-03 11:14:07.029916 7fb269f70700 client.5562 cap_delay_requeue on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.029956 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 312 to dispatch throttler 312/104857600 2011-11-03 11:14:07.029979 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41b68150 2011-11-03 11:14:07.030005 7fb269f70700 client.5562 cap mds.0 issued pAsLsXsFcb implemented pAsLsXsFcb revoking - 2011-11-03 11:14:07.030023 7fb269f70700 client.5562 delaying cap release 2011-11-03 11:14:07.030046 7fb269f70700 client.5562 put_inode on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.030059 7fb269f70700 inode.put on 0x4188e010 10000000981.head now 1 2011-11-03 11:14:07.030071 7fb269f70700 client.5562 close exit(14) 2011-11-03 11:14:07.035003 7fb269f70700 client.5562 lstat enter (relpath/user/nwatkins/in-dir/_partition.lst mask 341) 2011-11-03 11:14:07.035082 7fb269f70700 client.5562 path_walk #1/user/nwatkins/in-dir/_partition.lst 2011-11-03 11:14:07.035117 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) user 2011-11-03 11:14:07.035136 7fb269f70700 client.5562 _lookup have dn user mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:07.035174 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) user = 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) 2011-11-03 11:14:07.035196 7fb269f70700 client.5562 1 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) nwatkins 2011-11-03 11:14:07.035212 7fb269f70700 client.5562 _lookup have dn nwatkins mds.0 ttl 2011-11-03 11:14:37.012301 seq 5 2011-11-03 11:14:07.035245 7fb269f70700 client.5562 _lookup 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) nwatkins = 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) 2011-11-03 11:14:07.035267 7fb269f70700 client.5562 2 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) in-dir 2011-11-03 11:14:07.035281 7fb269f70700 client.5562 _lookup have dn in-dir mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:07.035316 7fb269f70700 client.5562 _lookup 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) in-dir = 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) 2011-11-03 11:14:07.035339 7fb269f70700 client.5562 3 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) _partition.lst 2011-11-03 11:14:07.035353 7fb269f70700 client.5562 _lookup have dn _partition.lst mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:07.035391 7fb269f70700 client.5562 _lookup 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) _partition.lst = 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.035412 7fb269f70700 client.5562 _getattr mask pAsLsXsFs issued=0 2011-11-03 11:14:07.035444 7fb269f70700 client.5562 choose_target_mds starting with req->inode 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.035497 7fb269f70700 client.5562 choose_target_mds 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) is_hash=0 hash=0 2011-11-03 11:14:07.035521 7fb269f70700 client.5562 choose_target_mds from caps on inode 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.035534 7fb269f70700 client.5562 mds is 0 2011-11-03 11:14:07.035547 7fb269f70700 client.5562 send_request rebuilding request 16 for mds.0 2011-11-03 11:14:07.035563 7fb269f70700 client.5562 encode_cap_releases enter (req: 0x41b68150, mds: 0) 2011-11-03 11:14:07.035578 7fb269f70700 client.5562 send_request set sent_stamp to 2011-11-03 11:14:07.035574 2011-11-03 11:14:07.035597 7fb269f70700 client.5562 send_request client_request(unknown.0:16 getattr pAsLsXsFs #10000000981) v1 to mds.0 2011-11-03 11:14:07.035624 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5562:16 getattr pAsLsXsFs #10000000981) v1 -- ?+0 0x4188a5b0 2011-11-03 11:14:07.035648 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_request(client.5562:16 getattr pAsLsXsFs #10000000981) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:07.035677 7fb269f70700 client.5562 awaiting reply|forward|kick on 0x7fb269f6e640 2011-11-03 11:14:07.035710 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.035742 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 19 0x4188a5b0 client_request(client.5562:16 getattr pAsLsXsFs #10000000981) v1 2011-11-03 11:14:07.035765 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 19 0x4188a5b0 2011-11-03 11:14:07.035782 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x4188a5b0 2011-11-03 11:14:07.035809 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.035826 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.089424 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:07.089461 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.089483 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:07.089508 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=26 src mds.0 front=312 data=0 off 0 2011-11-03 11:14:07.089528 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 312 from dispatch throttler 0/104857600 2011-11-03 11:14:07.089553 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 312 2011-11-03 11:14:07.089576 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:07.089598 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 312 + 0 + 0 byte message 2011-11-03 11:14:07.089635 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 19 0x4188ac40 client_reply(???:16 = 0 Success) v1 2011-11-03 11:14:07.089659 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:07.089686 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.089717 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.089745 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 19 2011-11-03 11:14:07.089771 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:07.089803 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x4188ac40 2011-11-03 11:14:07.089836 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 19 ==== client_reply(???:16 = 0 Success) v1 ==== 312+0+0 (4248540504 0 0) 0x4188ac40 con 0x41c02110 2011-11-03 11:14:07.089854 7fb267c69700 client.5562 handle_client_reply got a reply. Safe:1 tid 16 2011-11-03 11:14:07.089872 7fb267c69700 client.5562 insert_trace from 2011-11-03 11:14:07.035574 mds.0 is_target=1 is_dentry=0 2011-11-03 11:14:07.089887 7fb267c69700 client.5562 features 0xffa 2011-11-03 11:14:07.089903 7fb267c69700 client.5562 update_snap_trace len 48 2011-11-03 11:14:07.089922 7fb267c69700 client.5562 update_snap_trace snaprealm(1 nref=15 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:07.089940 7fb267c69700 client.5562 hrm is_target=1 is_dentry=0 2011-11-03 11:14:07.089975 7fb267c69700 client.5562 add_update_inode had 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) caps pAsLsXsFscrb 2011-11-03 11:14:07.090010 7fb267c69700 client.5562 update_inode_file_bits 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFcb(0=pAsLsXsFcb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) pAsLsXsFcb mtime 2011-11-03 11:14:01.196477 2011-11-03 11:14:07.090045 7fb267c69700 client.5562 add_update_cap issued pAsLsXsFcb -> pAsLsXsFscrb from mds.0 on 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.090063 7fb267c69700 client.5562 handle_client_reply signalling caller 0x7fb269f6e640 2011-11-03 11:14:07.090083 7fb267c69700 client.5562 handle_client_reply awaiting kickback on tid 16 0x7fb267c68490 2011-11-03 11:14:07.090111 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.090137 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.090161 7fb269f70700 client.5562 sendrecv kickback on tid 16 0x7fb267c68490 2011-11-03 11:14:07.090191 7fb269f70700 client.5562 lat 0.054613 2011-11-03 11:14:07.090221 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=784 src mds.0 front=180 data=0 off 0 2011-11-03 11:14:07.090248 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 180 from dispatch throttler 312/104857600 2011-11-03 11:14:07.090293 7fb269f70700 client.5562 _getattr result=0 2011-11-03 11:14:07.090322 7fb269f70700 client.5562 fill_stat on 10000000981 snap/devhead mode 0100777 mtime 2011-11-03 11:14:01.196477 ctime 2011-11-03 11:14:01.196477 2011-11-03 11:14:07.090343 7fb269f70700 client.5562 lstat exit (relpath/user/nwatkins/in-dir/_partition.lst mask 341) 2011-11-03 11:14:07.090370 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 180 2011-11-03 11:14:07.090400 7fb269f70700 client.5562 open enter(/user/nwatkins/in-dir/_partition.lst, 0,0) = 2011-11-03 11:14:07.090428 7fb269f70700 client.5562 path_walk #1/user/nwatkins/in-dir/_partition.lst 2011-11-03 11:14:07.090457 7fb269f70700 client.5562 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) user 2011-11-03 11:14:07.090477 7fb269f70700 client.5562 _lookup have dn user mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:07.090504 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 312 to dispatch throttler 492/104857600 2011-11-03 11:14:07.090527 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x4188ac40 2011-11-03 11:14:07.090568 7fb269f70700 client.5562 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) user = 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) 2011-11-03 11:14:07.090599 7fb269f70700 client.5562 1 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) nwatkins 2011-11-03 11:14:07.090619 7fb269f70700 client.5562 _lookup have dn nwatkins mds.0 ttl 2011-11-03 11:14:37.012301 seq 5 2011-11-03 11:14:07.090658 7fb269f70700 client.5562 _lookup 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) nwatkins = 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) 2011-11-03 11:14:07.090686 7fb269f70700 client.5562 2 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) in-dir 2011-11-03 11:14:07.090705 7fb269f70700 client.5562 _lookup have dn in-dir mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:07.090734 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:07.090761 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 180 + 0 + 0 byte message 2011-11-03 11:14:07.090803 7fb269f70700 client.5562 _lookup 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) in-dir = 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) 2011-11-03 11:14:07.090835 7fb269f70700 client.5562 3 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) _partition.lst 2011-11-03 11:14:07.090853 7fb269f70700 client.5562 _lookup have dn _partition.lst mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:07.090893 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 20 0x4188ea60 client_caps(grant ino 10000000981 6765 seq 5 caps=pAsLsXsFscrb dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v2 2011-11-03 11:14:07.090930 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:07.090992 7fb269f70700 client.5562 _lookup 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) _partition.lst = 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.091027 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.091061 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.091090 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 20 2011-11-03 11:14:07.091117 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:07.091159 7fb269f70700 client.5562 check_caps on 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) wanted pFscr used - is_delayed=1 2011-11-03 11:14:07.091190 7fb269f70700 client.5562 cap mds.0 issued pAsLsXsFscrb implemented pAsLsXsFscrb revoking - 2011-11-03 11:14:07.091230 7fb269f70700 client.5562 send_cap 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) mds.0 seq 5 used - want pFscr flush - retain pAsxLsxXsxFsxcrwbl held pAsLsXsFscrb revoking - dropping - 2011-11-03 11:14:07.091249 7fb269f70700 client.5562 auth cap, setting max_size = 0 2011-11-03 11:14:07.091283 7fb269f70700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 10000000981 6765 seq 5 caps=pAsLsXsFscrb dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 -- ?+0 0x4188edd0 2011-11-03 11:14:07.091309 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x4188ea60 2011-11-03 11:14:07.091349 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 20 ==== client_caps(grant ino 10000000981 6765 seq 5 caps=pAsLsXsFscrb dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v2 ==== 180+0+0 (3731982273 0 0) 0x4188ea60 con 0x41c02110 2011-11-03 11:14:07.091376 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=784 src mds.0 front=180 data=0 off 0 2011-11-03 11:14:07.091402 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 180 from dispatch throttler 180/104857600 2011-11-03 11:14:07.091432 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.091457 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.091483 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 180 2011-11-03 11:14:07.091523 7fb269f70700 -- 192.168.141.130:0/2011511 submit_message client_caps(update ino 10000000981 6765 seq 5 caps=pAsLsXsFscrb dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:07.091554 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:07.091594 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 180 + 0 + 0 byte message 2011-11-03 11:14:07.091622 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.091662 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 20 0x4188edd0 client_caps(update ino 10000000981 6765 seq 5 caps=pAsLsXsFscrb dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 2011-11-03 11:14:07.091689 7fb269f70700 inode.get on 0x4188e010 10000000981.head now 2 2011-11-03 11:14:07.091713 7fb269f70700 client.5562 _create_fh 10000000981 mode 1 2011-11-03 11:14:07.091731 7fb269f70700 client.5562 trim_cache size 12 max 16384 2011-11-03 11:14:07.091756 7fb269f70700 client.5562 open exit(#1/user/nwatkins/in-dir/_partition.lst, 0) = 15 2011-11-03 11:14:07.091777 7fb269f70700 client.5562 describe_layout(15) = 0 2011-11-03 11:14:07.091800 7fb269f70700 client.5562 close enter(15) 2011-11-03 11:14:07.091830 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 20 0x4188edd0 2011-11-03 11:14:07.091858 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x4188edd0 2011-11-03 11:14:07.091882 7fb267c69700 client.5562 mds.0 seq now 2 2011-11-03 11:14:07.091910 7fb267c69700 client.5562 handle_cap_grant on in 10000000981 mds.0 seq 5 caps now pAsLsXsFscrb was pAsLsXsFscrb 2011-11-03 11:14:07.091947 7fb267c69700 client.5562 update_inode_file_bits 10000000981.head(ref=2 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) pAsLsXsFscrb mtime 2011-11-03 11:14:01.196477 2011-11-03 11:14:07.091968 7fb267c69700 client.5562 caps unchanged at pAsLsXsFscrb 2011-11-03 11:14:07.092007 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 21 0x41887230 client_caps(revoke ino 10000000981 6765 seq 6 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v2 2011-11-03 11:14:07.092040 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:07.092065 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.092092 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 180 to dispatch throttler 360/104857600 2011-11-03 11:14:07.092113 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x4188ea60 2011-11-03 11:14:07.092140 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.092169 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 21 2011-11-03 11:14:07.092206 7fb269f70700 client.5562 _release_fh 0x418885c0 mode 1 on 10000000981.head(ref=2 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.092240 7fb269f70700 client.5562 _flush 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.092257 7fb269f70700 client.5562 nothing to flush 2011-11-03 11:14:07.092295 7fb269f70700 client.5562 check_caps on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) wanted - used - is_delayed=0 2011-11-03 11:14:07.092334 7fb269f70700 client.5562 cap_delay_requeue on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.092362 7fb269f70700 client.5562 cap mds.0 issued pAsLsXsFscrb implemented pAsLsXsFscrb revoking - 2011-11-03 11:14:07.092379 7fb269f70700 client.5562 delaying cap release 2011-11-03 11:14:07.092408 7fb269f70700 client.5562 put_inode on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) 2011-11-03 11:14:07.092426 7fb269f70700 inode.put on 0x4188e010 10000000981.head now 1 2011-11-03 11:14:07.092444 7fb269f70700 client.5562 close exit(15) 2011-11-03 11:14:07.092468 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x41887230 2011-11-03 11:14:07.092507 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 21 ==== client_caps(revoke ino 10000000981 6765 seq 6 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v2 ==== 180+0+0 (540916426 0 0) 0x41887230 con 0x41c02110 2011-11-03 11:14:07.092525 7fb267c69700 client.5562 mds.0 seq now 3 2011-11-03 11:14:07.092548 7fb267c69700 client.5562 handle_cap_grant on in 10000000981 mds.0 seq 6 caps now pAsLsXsFscr was pAsLsXsFscrb 2011-11-03 11:14:07.092583 7fb267c69700 client.5562 update_inode_file_bits 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscrb(0=pAsLsXsFscrb) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) pAsLsXsFscrb mtime 2011-11-03 11:14:01.196477 2011-11-03 11:14:07.092603 7fb267c69700 client.5562 revocation of Fb 2011-11-03 11:14:07.092635 7fb267c69700 client.5562 check_caps on 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) wanted - used - is_delayed=1 2011-11-03 11:14:07.092661 7fb267c69700 client.5562 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscrb revoking Fb 2011-11-03 11:14:07.092678 7fb267c69700 client.5562 completed revocation of Fb 2011-11-03 11:14:07.092715 7fb267c69700 client.5562 send_cap 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) mds.0 seq 6 used - want - flush - retain pAsLsXsFs held pAsLsXsFscrb revoking Fb dropping Fcr 2011-11-03 11:14:07.092734 7fb267c69700 client.5562 auth cap, setting max_size = 0 2011-11-03 11:14:07.092763 7fb267c69700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 10000000981 6765 seq 6 caps=pAsLsXsFs dirty=- wanted=- follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 -- ?+0 0x4188ea60 2011-11-03 11:14:07.092794 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.092817 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.092862 7fb267c69700 -- 192.168.141.130:0/2011511 submit_message client_caps(update ino 10000000981 6765 seq 6 caps=pAsLsXsFs dirty=- wanted=- follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:07.092896 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 180 to dispatch throttler 180/104857600 2011-11-03 11:14:07.092924 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x41887230 2011-11-03 11:14:07.092953 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.092991 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 21 0x4188ea60 client_caps(update ino 10000000981 6765 seq 6 caps=pAsLsXsFs dirty=- wanted=- follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 2011-11-03 11:14:07.093014 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 21 0x4188ea60 2011-11-03 11:14:07.093031 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x4188ea60 2011-11-03 11:14:07.093058 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.093074 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.299995 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:07.300028 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.300051 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:07.300075 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.722660 7fb268c6b700 client.5562 renew_caps() 2011-11-03 11:14:07.722734 7fb268c6b700 client.5562 renew_caps requesting from mds.0 2011-11-03 11:14:07.722749 7fb268c6b700 client.5562 renew_caps mds.0 2011-11-03 11:14:07.722797 7fb268c6b700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_renewcaps seq 2) v1 -- ?+0 0x41dec340 2011-11-03 11:14:07.722819 7fb268c6b700 -- 192.168.141.130:0/2011511 submit_message client_session(request_renewcaps seq 2) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:07.722865 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.722895 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 22 0x41dec340 client_session(request_renewcaps seq 2) v1 2011-11-03 11:14:07.722921 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 22 0x41dec340 2011-11-03 11:14:07.722939 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41dec340 2011-11-03 11:14:07.722968 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.722984 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.723172 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:07.723198 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.723221 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:07.723264 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:14:07.723285 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:14:07.723309 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 28 2011-11-03 11:14:07.723332 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:07.723349 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:14:07.723378 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 22 0x4188a5b0 client_session(renewcaps seq 2) v1 2011-11-03 11:14:07.723402 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:07.723434 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x4188a5b0 2011-11-03 11:14:07.723466 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 22 ==== client_session(renewcaps seq 2) v1 ==== 28+0+0 (1357429227 0 0) 0x4188a5b0 con 0x41c02110 2011-11-03 11:14:07.723485 7fb267c69700 client.5562 handle_client_session client_session(renewcaps seq 2) v1 2011-11-03 11:14:07.723510 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:07.723548 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.723573 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 22 2011-11-03 11:14:07.723598 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:07.723618 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:07.723645 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:14:07.723664 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x4188a5b0 2011-11-03 11:14:09.507934 7fb269f70700 client.5557 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:09.507984 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:09.508017 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:09.508036 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.508074 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:09.508097 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:09.508111 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.508146 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:09.508188 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:09.508202 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.508237 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:09.508258 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:09.508273 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:09.508307 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:09.508331 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:09.508345 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:09.508377 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:09.508397 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:09.508412 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:09.508448 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.508468 7fb269f70700 client.5557 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.508486 7fb269f70700 client.5557 fill_stat on 100000000b2 snap/devhead mode 0100777 mtime 2011-11-03 11:14:05.788394 ctime 2011-11-03 11:14:05.785578 2011-11-03 11:14:09.508500 7fb269f70700 client.5557 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:09.508524 7fb269f70700 client.5557 open enter(/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0,0) = 2011-11-03 11:14:09.508541 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:09.508562 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:09.508581 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.508686 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:09.508709 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:09.508723 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.508757 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:09.508779 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:09.508793 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.508826 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:09.508848 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:09.508863 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:09.508894 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:09.508915 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:09.508930 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:09.508960 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:09.508981 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:09.508995 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:09.509030 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.509063 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted pFscr used Fc is_delayed=1 2011-11-03 11:14:09.509097 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.509112 7fb269f70700 inode.get on 0x41899360 100000000b2.head now 2 2011-11-03 11:14:09.509125 7fb269f70700 client.5557 _create_fh 100000000b2 mode 1 2011-11-03 11:14:09.509137 7fb269f70700 client.5557 trim_cache size 6 max 16384 2011-11-03 11:14:09.509157 7fb269f70700 client.5557 open exit(#1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0) = 14 2011-11-03 11:14:09.509171 7fb269f70700 client.5557 describe_layout(14) = 0 2011-11-03 11:14:09.509186 7fb269f70700 client.5557 close enter(14) 2011-11-03 11:14:09.509212 7fb269f70700 client.5557 _release_fh 0x41877cd0 mode 1 on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.509238 7fb269f70700 client.5557 _flush 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.509250 7fb269f70700 client.5557 nothing to flush 2011-11-03 11:14:09.509276 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted Fc used Fc is_delayed=0 2011-11-03 11:14:09.509302 7fb269f70700 client.5557 cap_delay_requeue on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.509323 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.509335 7fb269f70700 client.5557 delaying cap release 2011-11-03 11:14:09.509359 7fb269f70700 client.5557 put_inode on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.509372 7fb269f70700 inode.put on 0x41899360 100000000b2.head now 1 2011-11-03 11:14:09.509385 7fb269f70700 client.5557 close exit(14) 2011-11-03 11:14:09.510553 7fb269f70700 client.5557 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:09.510588 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:09.510616 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:09.510632 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.510669 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:09.510691 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:09.510705 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.510758 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:09.510780 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:09.510794 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.510828 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:09.510849 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:09.510864 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:09.510896 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:09.510917 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:09.510932 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:09.510962 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:09.510982 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:09.510997 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:09.511032 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.511051 7fb269f70700 client.5557 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.511067 7fb269f70700 client.5557 fill_stat on 100000000b2 snap/devhead mode 0100777 mtime 2011-11-03 11:14:05.788394 ctime 2011-11-03 11:14:05.785578 2011-11-03 11:14:09.511082 7fb269f70700 client.5557 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:09.511103 7fb269f70700 client.5557 open enter(/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0,0) = 2011-11-03 11:14:09.511120 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:09.511142 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:09.511170 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.511205 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:09.511227 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:09.511240 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.511274 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:09.511296 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:09.511310 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.511343 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:09.511365 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:09.511379 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:09.511410 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:09.511432 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:09.511446 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:09.511476 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:09.511497 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:09.511511 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:09.511545 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.511589 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted pFscr used Fc is_delayed=1 2011-11-03 11:14:09.511609 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.511623 7fb269f70700 inode.get on 0x41899360 100000000b2.head now 2 2011-11-03 11:14:09.511636 7fb269f70700 client.5557 _create_fh 100000000b2 mode 1 2011-11-03 11:14:09.511648 7fb269f70700 client.5557 trim_cache size 6 max 16384 2011-11-03 11:14:09.511665 7fb269f70700 client.5557 open exit(#1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0) = 15 2011-11-03 11:14:09.511679 7fb269f70700 client.5557 describe_layout(15) = 0 2011-11-03 11:14:09.511693 7fb269f70700 client.5557 close enter(15) 2011-11-03 11:14:09.511717 7fb269f70700 client.5557 _release_fh 0x41877cd0 mode 1 on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.511742 7fb269f70700 client.5557 _flush 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.511754 7fb269f70700 client.5557 nothing to flush 2011-11-03 11:14:09.511779 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted Fc used Fc is_delayed=0 2011-11-03 11:14:09.511803 7fb269f70700 client.5557 cap_delay_requeue on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.511824 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.511837 7fb269f70700 client.5557 delaying cap release 2011-11-03 11:14:09.511860 7fb269f70700 client.5557 put_inode on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.511873 7fb269f70700 inode.put on 0x41899360 100000000b2.head now 1 2011-11-03 11:14:09.511885 7fb269f70700 client.5557 close exit(15) 2011-11-03 11:14:09.511926 7fb269f70700 client.5557 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:09.511943 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:09.511965 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:09.511978 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.512014 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:09.512036 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:09.512064 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.512099 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:09.512121 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:09.512134 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.512169 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:09.512190 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:09.512204 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:09.512236 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:09.512257 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:09.512271 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:09.512301 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:09.512321 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:09.512336 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:09.512369 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.512386 7fb269f70700 client.5557 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.512402 7fb269f70700 client.5557 fill_stat on 100000000b2 snap/devhead mode 0100777 mtime 2011-11-03 11:14:05.788394 ctime 2011-11-03 11:14:05.785578 2011-11-03 11:14:09.512417 7fb269f70700 client.5557 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:09.512466 7fb269f70700 client.5557 open enter(/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0,0) = 2011-11-03 11:14:09.512483 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:09.512511 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:09.512533 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.512567 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:09.512589 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:09.512604 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.512638 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:09.512660 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:09.512673 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.512707 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:09.512728 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:09.512743 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:09.512774 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:09.512796 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:09.512810 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:09.512840 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:09.512861 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:09.512875 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:09.512910 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.512952 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted pFscr used Fc is_delayed=1 2011-11-03 11:14:09.512971 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.512984 7fb269f70700 inode.get on 0x41899360 100000000b2.head now 2 2011-11-03 11:14:09.512996 7fb269f70700 client.5557 _create_fh 100000000b2 mode 1 2011-11-03 11:14:09.513008 7fb269f70700 client.5557 trim_cache size 6 max 16384 2011-11-03 11:14:09.513023 7fb269f70700 client.5557 open exit(#1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0) = 16 2011-11-03 11:14:09.513053 7fb269f70700 client.5557 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:09.513069 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:09.513090 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:09.513104 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.513138 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:09.513160 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:09.513174 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.513208 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:09.513230 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:09.513243 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.513276 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:09.513297 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:09.513312 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:09.513343 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:09.513364 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:09.513397 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:09.513429 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:09.513450 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:09.513464 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:09.513498 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.513516 7fb269f70700 client.5557 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.513531 7fb269f70700 client.5557 fill_stat on 100000000b2 snap/devhead mode 0100777 mtime 2011-11-03 11:14:05.788394 ctime 2011-11-03 11:14:05.785578 2011-11-03 11:14:09.513545 7fb269f70700 client.5557 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:09.513564 7fb269f70700 client.5557 open enter(/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0,0) = 2011-11-03 11:14:09.513579 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:09.513602 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:09.513616 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.513649 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:09.513671 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:09.513685 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.513719 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:09.513741 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:09.513754 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.513787 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:09.513822 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:09.513837 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:09.513868 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:09.513890 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:09.513905 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:09.513935 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:09.513956 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:09.513970 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:09.514004 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.514034 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=2} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted pFscr used Fc is_delayed=1 2011-11-03 11:14:09.514053 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.514066 7fb269f70700 inode.get on 0x41899360 100000000b2.head now 3 2011-11-03 11:14:09.514078 7fb269f70700 client.5557 _create_fh 100000000b2 mode 1 2011-11-03 11:14:09.514089 7fb269f70700 client.5557 trim_cache size 6 max 16384 2011-11-03 11:14:09.514105 7fb269f70700 client.5557 open exit(#1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken, 0) = 17 2011-11-03 11:14:09.514118 7fb269f70700 client.5557 describe_layout(17) = 0 2011-11-03 11:14:09.514130 7fb269f70700 client.5557 close enter(17) 2011-11-03 11:14:09.514154 7fb269f70700 client.5557 _release_fh 0x41877d60 mode 1 on 100000000b2.head(ref=3 cap_refs={1024=-1,2048=0,8192=0} open={1=2} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.514178 7fb269f70700 client.5557 put_inode on 100000000b2.head(ref=3 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.514191 7fb269f70700 inode.put on 0x41899360 100000000b2.head now 2 2011-11-03 11:14:09.514203 7fb269f70700 client.5557 close exit(17) 2011-11-03 11:14:09.514236 7fb269f70700 client.5557 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:09.514266 7fb269f70700 client.5557 path_walk #1/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken 2011-11-03 11:14:09.514289 7fb269f70700 client.5557 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp 2011-11-03 11:14:09.514302 7fb269f70700 client.5557 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.514337 7fb269f70700 client.5557 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:09.514359 7fb269f70700 client.5557 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins 2011-11-03 11:14:09.514373 7fb269f70700 client.5557 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.514407 7fb269f70700 client.5557 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:09.514429 7fb269f70700 client.5557 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred 2011-11-03 11:14:09.514442 7fb269f70700 client.5557 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.514476 7fb269f70700 client.5557 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:09.514497 7fb269f70700 client.5557 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system 2011-11-03 11:14:09.514512 7fb269f70700 client.5557 _lookup have dn system mds.0 ttl 2011-11-03 11:14:36.481174 seq 2 2011-11-03 11:14:09.514543 7fb269f70700 client.5557 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) system = 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:09.514564 7fb269f70700 client.5557 4 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 2011-11-03 11:14:09.514579 7fb269f70700 client.5557 _lookup have dn job_201111031113_0001 mds.0 ttl 2011-11-03 11:14:36.482785 seq 3 2011-11-03 11:14:09.514609 7fb269f70700 client.5557 _lookup 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) job_201111031113_0001 = 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:09.514629 7fb269f70700 client.5557 5 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken 2011-11-03 11:14:09.514644 7fb269f70700 client.5557 _lookup have dn jobToken mds.0 ttl 2011-11-03 11:14:36.484537 seq 4 2011-11-03 11:14:09.514678 7fb269f70700 client.5557 _lookup 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) jobToken = 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.514707 7fb269f70700 client.5557 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.514723 7fb269f70700 client.5557 fill_stat on 100000000b2 snap/devhead mode 0100777 mtime 2011-11-03 11:14:05.788394 ctime 2011-11-03 11:14:05.785578 2011-11-03 11:14:09.514738 7fb269f70700 client.5557 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/system/job_201111031113_0001/jobToken mask 341) 2011-11-03 11:14:09.517569 7fb269f70700 client.5557 get_caps 100000000b2.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:09.517590 7fb269f70700 client.5557 lock_fh_pos 0x41877cd0 2011-11-03 11:14:09.517617 7fb269f70700 client.5557 _read_async 100000000b2.head(ref=2 cap_refs={1024=-1,2048=1,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 0~4096 2011-11-03 11:14:09.517632 7fb269f70700 client.5557 readahead=0 nr_consec=0 max_byes=0 max_periods=1 2011-11-03 11:14:09.517688 7fb269f70700 client.5557 unlock_fh_pos 0x41877cd0 2011-11-03 11:14:09.517702 7fb269f70700 client.5557 readahead nr_consec_read 1 for 106 bytes .. last_pos 0 .. offset 0 2011-11-03 11:14:09.517717 7fb269f70700 client.5557 read(16, 0x41887230, 4096, -1) = 106 2011-11-03 11:14:09.517759 7fb269f70700 client.5557 get_caps 100000000b2.head(ref=2 cap_refs={1024=-2,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:09.517773 7fb269f70700 client.5557 lock_fh_pos 0x41877cd0 2011-11-03 11:14:09.517798 7fb269f70700 client.5557 _read_async 100000000b2.head(ref=2 cap_refs={1024=-2,2048=1,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 106~4096 2011-11-03 11:14:09.517810 7fb269f70700 client.5557 unlock_fh_pos 0x41877cd0 2011-11-03 11:14:09.517822 7fb269f70700 client.5557 readahead nr_consec_read 2 for 106 bytes .. last_pos 106 .. offset 106 2011-11-03 11:14:09.517835 7fb269f70700 client.5557 read(16, 0x41887230, 4096, -1) = 0 2011-11-03 11:14:09.518004 7fb269f70700 client.5557 _fsync(0x41877cd0, data+metadata) 2011-11-03 11:14:09.518033 7fb269f70700 client.5557 _flush 100000000b2.head(ref=2 cap_refs={1024=-3,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.518046 7fb269f70700 client.5557 nothing to flush 2011-11-03 11:14:09.518057 7fb269f70700 client.5557 no metadata needs to commit 2011-11-03 11:14:09.518070 7fb269f70700 client.5557 wait_sync_caps want 0 (last is 0, 0 total flushing) 2011-11-03 11:14:09.518083 7fb269f70700 client.5557 ino 100000000b2 has no uncommitted writes 2011-11-03 11:14:09.518096 7fb269f70700 client.5557 fsync(16, 0) = 0 2011-11-03 11:14:09.518115 7fb269f70700 client.5557 close enter(16) 2011-11-03 11:14:09.518140 7fb269f70700 client.5557 _release_fh 0x41877cd0 mode 1 on 100000000b2.head(ref=2 cap_refs={1024=-3,2048=0,8192=0} open={1=1} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.518171 7fb269f70700 client.5557 _flush 100000000b2.head(ref=2 cap_refs={1024=-3,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.518191 7fb269f70700 client.5557 nothing to flush 2011-11-03 11:14:09.518218 7fb269f70700 client.5557 check_caps on 100000000b2.head(ref=2 cap_refs={1024=-3,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted Fc used Fc is_delayed=0 2011-11-03 11:14:09.518243 7fb269f70700 client.5557 cap_delay_requeue on 100000000b2.head(ref=2 cap_refs={1024=-3,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.518264 7fb269f70700 client.5557 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.518277 7fb269f70700 client.5557 delaying cap release 2011-11-03 11:14:09.518300 7fb269f70700 client.5557 put_inode on 100000000b2.head(ref=2 cap_refs={1024=-3,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) 2011-11-03 11:14:09.518313 7fb269f70700 inode.put on 0x41899360 100000000b2.head now 1 2011-11-03 11:14:09.518325 7fb269f70700 client.5557 close exit(16) 2011-11-03 11:14:09.522844 7fb269f70700 -- :/0 register_entity client.? 2011-11-03 11:14:09.522887 7fb269f70700 -- :/0 register_entity client.? at :/0 2011-11-03 11:14:09.522929 7fb269f70700 -- :/0 messenger.start 2011-11-03 11:14:09.523003 7fb26565f700 -- :/3011511 reaper_entry start 2011-11-03 11:14:09.523071 7fb26565f700 -- :/3011511 reaper 2011-11-03 11:14:09.523092 7fb26565f700 -- :/3011511 reaper done 2011-11-03 11:14:09.523116 7fb269f70700 -- :/3011511 ready :/3011511 2011-11-03 11:14:09.523480 7fb269f70700 -- :/3011511 connect_rank to 192.168.141.123:6789/0, creating pipe and registering 2011-11-03 11:14:09.523562 7fb269f70700 -- :/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=-1 pgs=0 cs=0 l=0).register_pipe 2011-11-03 11:14:09.523614 7fb262659700 -- :/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=-1 pgs=0 cs=0 l=0).writer: state = 1 policy.server=0 2011-11-03 11:14:09.523655 7fb262659700 -- :/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=-1 pgs=0 cs=0 l=0).connect 0 2011-11-03 11:14:09.523683 7fb269f70700 -- :/3011511 --> 192.168.141.123:6789/0 -- auth(proto 0 30 bytes) v1 -- ?+0 0x41cc16f0 con 0x41cc13a0 2011-11-03 11:14:09.523710 7fb269f70700 -- :/3011511 submit_message auth(proto 0 30 bytes) v1 remote 192.168.141.123:6789/0 2011-11-03 11:14:09.523738 7fb262659700 -- :/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=0 cs=0 l=0).connecting to 192.168.141.123:6789/0 2011-11-03 11:14:09.524302 7fb262659700 -- :/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=0 cs=0 l=0).connect read peer addr 192.168.141.123:6789/0 on socket 87 2011-11-03 11:14:09.524325 7fb262659700 -- :/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=0 cs=0 l=0).connect peer addr for me is 192.168.141.130:34026/0 2011-11-03 11:14:09.524341 7fb262659700 -- 192.168.141.130:0/3011511 learned my addr 192.168.141.130:0/3011511 2011-11-03 11:14:09.524381 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=0 cs=0 l=0).connect sent my addr 192.168.141.130:0/3011511 2011-11-03 11:14:09.524400 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=0 cs=0 l=0).connect sending gseq=1 cseq=0 proto=15 2011-11-03 11:14:09.524425 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=0 cs=0 l=0).connect wrote (self +) cseq, waiting for reply 2011-11-03 11:14:09.524538 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=0 cs=0 l=0).connect got reply tag 1 connect_seq 1 global_seq 381 proto 15 flags 1 2011-11-03 11:14:09.524572 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).connect success 1, lossy = 1, features 4090 2011-11-03 11:14:09.524591 7fb262659700 -- 192.168.141.130:0/3011511 >> :/0 pipe(0x41ae1990 sd=-1 pgs=0 cs=0 l=0).queue_received queuing pipe 2011-11-03 11:14:09.524618 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).connect starting reader 2011-11-03 11:14:09.524654 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x41ae1990 dequeued 0 2011-11-03 11:14:09.524692 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.524724 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer encoding 1 0x41cc16f0 auth(proto 0 30 bytes) v1 2011-11-03 11:14:09.524750 7fb263e5c700 client.-1 ms_handle_connect on 192.168.141.123:6789/0 2011-11-03 11:14:09.524784 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sending 1 0x41cc16f0 2011-11-03 11:14:09.524810 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_message 0x41cc16f0 2011-11-03 11:14:09.524842 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.524859 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.524885 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.525241 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got ACK 2011-11-03 11:14:09.525265 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.525287 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got MSG 2011-11-03 11:14:09.525312 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got envelope type=18 src mon.0 front=33 data=0 off 0 2011-11-03 11:14:09.525330 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader wants 33 from dispatch throttler 0/104857600 2011-11-03 11:14:09.525354 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got front 33 2011-11-03 11:14:09.525378 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).aborted = 0 2011-11-03 11:14:09.525394 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got 33 + 0 + 0 byte message 2011-11-03 11:14:09.525437 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got message 1 0x41cc16f0 auth_reply(proto 2 0 Success) v1 2011-11-03 11:14:09.525461 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.525479 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_ack 1 2011-11-03 11:14:09.525499 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.525515 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.525550 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:09.525573 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.525599 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x41cc1130 dequeued 0x41cc16f0 2011-11-03 11:14:09.525636 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mon.0 192.168.141.123:6789/0 1 ==== auth_reply(proto 2 0 Success) v1 ==== 33+0+0 (660703680 0 0) 0x41cc16f0 con 0x41cc13a0 2011-11-03 11:14:09.525794 7fb263e5c700 -- 192.168.141.130:0/3011511 --> 192.168.141.123:6789/0 -- auth(proto 2 32 bytes) v1 -- ?+0 0x41cc16f0 con 0x41cc13a0 2011-11-03 11:14:09.525814 7fb263e5c700 -- 192.168.141.130:0/3011511 submit_message auth(proto 2 32 bytes) v1 remote 192.168.141.123:6789/0 2011-11-03 11:14:09.525839 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 33 to dispatch throttler 33/104857600 2011-11-03 11:14:09.525858 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc16f0 2011-11-03 11:14:09.525885 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.525909 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer encoding 2 0x41cc16f0 auth(proto 2 32 bytes) v1 2011-11-03 11:14:09.525932 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sending 2 0x41cc16f0 2011-11-03 11:14:09.525951 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_message 0x41cc16f0 2011-11-03 11:14:09.525979 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.525996 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.526429 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got ACK 2011-11-03 11:14:09.526499 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.526527 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got MSG 2011-11-03 11:14:09.526554 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got envelope type=18 src mon.0 front=206 data=0 off 0 2011-11-03 11:14:09.526573 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader wants 206 from dispatch throttler 0/104857600 2011-11-03 11:14:09.526603 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got front 206 2011-11-03 11:14:09.526627 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).aborted = 0 2011-11-03 11:14:09.526644 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got 206 + 0 + 0 byte message 2011-11-03 11:14:09.526699 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got message 2 0x41cc1ea0 auth_reply(proto 2 0 Success) v1 2011-11-03 11:14:09.526726 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:09.526755 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.526796 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x41cc1130 dequeued 0x41cc1ea0 2011-11-03 11:14:09.526846 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mon.0 192.168.141.123:6789/0 2 ==== auth_reply(proto 2 0 Success) v1 ==== 206+0+0 (3934216309 0 0) 0x41cc1ea0 con 0x41cc13a0 2011-11-03 11:14:09.526873 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.526896 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_ack 2 2011-11-03 11:14:09.526917 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.526934 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.526962 7fb263e5c700 -- 192.168.141.130:0/3011511 --> 192.168.141.123:6789/0 -- auth(proto 2 165 bytes) v1 -- ?+0 0x41cc1ea0 con 0x41cc13a0 2011-11-03 11:14:09.526984 7fb263e5c700 -- 192.168.141.130:0/3011511 submit_message auth(proto 2 165 bytes) v1 remote 192.168.141.123:6789/0 2011-11-03 11:14:09.527004 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 206 to dispatch throttler 206/104857600 2011-11-03 11:14:09.527022 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc1ea0 2011-11-03 11:14:09.527048 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.527072 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer encoding 3 0x41cc1ea0 auth(proto 2 165 bytes) v1 2011-11-03 11:14:09.527098 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sending 3 0x41cc1ea0 2011-11-03 11:14:09.527117 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_message 0x41cc1ea0 2011-11-03 11:14:09.527148 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.527165 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.527533 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got ACK 2011-11-03 11:14:09.527562 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.527584 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got MSG 2011-11-03 11:14:09.527607 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got envelope type=18 src mon.0 front=409 data=0 off 0 2011-11-03 11:14:09.527624 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader wants 409 from dispatch throttler 0/104857600 2011-11-03 11:14:09.527649 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got front 409 2011-11-03 11:14:09.527675 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).aborted = 0 2011-11-03 11:14:09.527692 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got 409 + 0 + 0 byte message 2011-11-03 11:14:09.527719 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got message 3 0x41cc2050 auth_reply(proto 2 0 Success) v1 2011-11-03 11:14:09.527742 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:09.527788 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.527818 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x41cc1130 dequeued 0x41cc2050 2011-11-03 11:14:09.527849 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mon.0 192.168.141.123:6789/0 3 ==== auth_reply(proto 2 0 Success) v1 ==== 409+0+0 (681768442 0 0) 0x41cc2050 con 0x41cc13a0 2011-11-03 11:14:09.527876 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.527899 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_ack 3 2011-11-03 11:14:09.527921 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.527942 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.527973 7fb263e5c700 -- 192.168.141.130:0/3011511 --> 192.168.141.123:6789/0 -- mon_subscribe({mdsmap=0+,monmap=0+,osdmap=0}) v1 -- ?+0 0x41cc18d0 con 0x41cc13a0 2011-11-03 11:14:09.527996 7fb263e5c700 -- 192.168.141.130:0/3011511 submit_message mon_subscribe({mdsmap=0+,monmap=0+,osdmap=0}) v1 remote 192.168.141.123:6789/0 2011-11-03 11:14:09.528026 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.528057 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer encoding 4 0x41cc18d0 mon_subscribe({mdsmap=0+,monmap=0+,osdmap=0}) v1 2011-11-03 11:14:09.528084 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 409 to dispatch throttler 409/104857600 2011-11-03 11:14:09.528107 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc2050 2011-11-03 11:14:09.528131 7fb269f70700 client.5563 mounted: have osdmap 0 and mdsmap 0 2011-11-03 11:14:09.528163 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sending 4 0x41cc18d0 2011-11-03 11:14:09.528189 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_message 0x41cc18d0 2011-11-03 11:14:09.528211 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.528239 7fb269f70700 client.5563 target mds.0 not active, waiting for new mdsmap 2011-11-03 11:14:09.528292 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.528314 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.528565 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got ACK 2011-11-03 11:14:09.528592 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.528614 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got MSG 2011-11-03 11:14:09.528637 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got envelope type=21 src mon.0 front=535 data=0 off 0 2011-11-03 11:14:09.528671 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader wants 535 from dispatch throttler 0/104857600 2011-11-03 11:14:09.528698 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got front 535 2011-11-03 11:14:09.528728 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).aborted = 0 2011-11-03 11:14:09.528754 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got 535 + 0 + 0 byte message 2011-11-03 11:14:09.528781 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got message 4 0x41bcba90 mdsmap(e 14) v1 2011-11-03 11:14:09.528810 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.528834 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_ack 4 2011-11-03 11:14:09.528859 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:09.528890 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.528916 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x41cc1130 dequeued 0x41bcba90 2011-11-03 11:14:09.528950 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mon.0 192.168.141.123:6789/0 4 ==== mdsmap(e 14) v1 ==== 535+0+0 (6102218 0 0) 0x41bcba90 con 0x41cc13a0 2011-11-03 11:14:09.528969 7fb263e5c700 client.5563 handle_mds_map epoch 14 2011-11-03 11:14:09.528995 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got MSG 2011-11-03 11:14:09.529021 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 535 to dispatch throttler 535/104857600 2011-11-03 11:14:09.529044 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41bcba90 2011-11-03 11:14:09.529071 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.529095 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.529119 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.529142 7fb269f70700 client.5563 opening session to mds.0 2011-11-03 11:14:09.529168 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_open) v1 -- ?+0 0x7fb27048fc00 2011-11-03 11:14:09.529193 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_session(request_open) v1 remote, 192.168.141.123:6802/19591, new pipe. 2011-11-03 11:14:09.529211 7fb269f70700 -- 192.168.141.130:0/3011511 connect_rank to 192.168.141.123:6802/19591, creating pipe and registering 2011-11-03 11:14:09.529238 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got envelope type=4 src mon.0 front=187 data=0 off 0 2011-11-03 11:14:09.529260 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader wants 187 from dispatch throttler 0/104857600 2011-11-03 11:14:09.529290 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got front 187 2011-11-03 11:14:09.529314 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).aborted = 0 2011-11-03 11:14:09.529334 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got 187 + 0 + 0 byte message 2011-11-03 11:14:09.529373 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=-1 pgs=0 cs=0 l=0).writer: state = 1 policy.server=0 2011-11-03 11:14:09.529405 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=-1 pgs=0 cs=0 l=0).connect 0 2011-11-03 11:14:09.529434 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got message 5 0x41ae3ed0 mon_map v1 2011-11-03 11:14:09.529485 7fb269f70700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=0 cs=0 l=0).register_pipe 2011-11-03 11:14:09.529523 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=0 cs=0 l=0).connecting to 192.168.141.123:6802/19591 2011-11-03 11:14:09.529558 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:09.529590 7fb269f70700 client.5563 waiting for session to mds.0 to open 2011-11-03 11:14:09.529623 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.529653 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_ack 5 2011-11-03 11:14:09.529713 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.529744 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x41cc1130 dequeued 0x41ae3ed0 2011-11-03 11:14:09.529778 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mon.0 192.168.141.123:6789/0 5 ==== mon_map v1 ==== 187+0+0 (2564923774 0 0) 0x41ae3ed0 con 0x41cc13a0 2011-11-03 11:14:09.529807 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got MSG 2011-11-03 11:14:09.529839 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.529867 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.529894 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got envelope type=41 src mon.0 front=6122 data=0 off 0 2011-11-03 11:14:09.529920 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader wants 6122 from dispatch throttler 187/104857600 2011-11-03 11:14:09.529946 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 187 to dispatch throttler 6309/104857600 2011-11-03 11:14:09.529970 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41ae3ed0 2011-11-03 11:14:09.529996 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got front 6122 2011-11-03 11:14:09.530024 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).aborted = 0 2011-11-03 11:14:09.530043 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got 6122 + 0 + 0 byte message 2011-11-03 11:14:09.530078 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got message 6 0x41ae3ed0 osd_map(16..16 src has 1..16) v2 2011-11-03 11:14:09.530101 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:09.530123 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.530149 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x41cc1130 dequeued 0x41ae3ed0 2011-11-03 11:14:09.530182 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mon.0 192.168.141.123:6789/0 6 ==== osd_map(16..16 src has 1..16) v2 ==== 6122+0+0 (849204460 0 0) 0x41ae3ed0 con 0x41cc13a0 2011-11-03 11:14:09.530214 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=0 cs=0 l=0).connect read peer addr 192.168.141.123:6802/19591 on socket 89 2011-11-03 11:14:09.530256 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=0 cs=0 l=0).connect peer addr for me is 192.168.141.130:54381/0 2011-11-03 11:14:09.530301 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.530329 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_ack 6 2011-11-03 11:14:09.530352 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.530372 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.530400 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got MSG 2011-11-03 11:14:09.530432 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 6122 to dispatch throttler 6122/104857600 2011-11-03 11:14:09.530457 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41ae3ed0 2011-11-03 11:14:09.530485 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got envelope type=16 src mon.0 front=20 data=0 off 0 2011-11-03 11:14:09.530510 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader wants 20 from dispatch throttler 0/104857600 2011-11-03 11:14:09.530545 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=0 cs=0 l=0).connect sent my addr 192.168.141.130:0/3011511 2011-11-03 11:14:09.530575 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got front 20 2011-11-03 11:14:09.530606 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).aborted = 0 2011-11-03 11:14:09.530622 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got 20 + 0 + 0 byte message 2011-11-03 11:14:09.530654 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=0 cs=0 l=0).connect.authorizer_len=174 protocol=2 2011-11-03 11:14:09.530683 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=0 cs=0 l=0).connect sending gseq=2 cseq=0 proto=32 2011-11-03 11:14:09.530710 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader got message 7 0x41ae5d20 mon_subscribe_ack(300s) v1 2011-11-03 11:14:09.530748 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=0 cs=0 l=0).connect wrote (self +) cseq, waiting for reply 2011-11-03 11:14:09.530780 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:09.530812 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x41cc1130 dequeued 0x41ae5d20 2011-11-03 11:14:09.530844 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mon.0 192.168.141.123:6789/0 7 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3759263596 0 0) 0x41ae5d20 con 0x41cc13a0 2011-11-03 11:14:09.530866 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 20 to dispatch throttler 20/104857600 2011-11-03 11:14:09.530884 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41ae5d20 2011-11-03 11:14:09.530911 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.530943 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.530965 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_ack 7 2011-11-03 11:14:09.530995 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.531024 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.531054 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=0 cs=0 l=0).connect got reply tag 1 connect_seq 1 global_seq 1180 proto 32 flags 0 2011-11-03 11:14:09.531077 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=0 cs=0 l=0).reply.authorizer_len=36 2011-11-03 11:14:09.531167 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).connect success 1, lossy = 0, features 4090 2011-11-03 11:14:09.531188 7fb262457700 -- 192.168.141.130:0/3011511 >> :/0 pipe(0x41ae1990 sd=-1 pgs=0 cs=0 l=0).queue_received queuing pipe 2011-11-03 11:14:09.531210 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).connect starting reader 2011-11-03 11:14:09.531235 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x41ae1990 dequeued 0 2011-11-03 11:14:09.531260 7fb263e5c700 client.5563 ms_handle_connect on 192.168.141.123:6802/19591 2011-11-03 11:14:09.531284 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.531315 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 1 0x7fb27048fc00 client_session(request_open) v1 2011-11-03 11:14:09.531349 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.531386 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 1 0x7fb27048fc00 2011-11-03 11:14:09.531410 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb27048fc00 2011-11-03 11:14:09.531440 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.531457 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.652253 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.652286 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.652308 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.652333 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:14:09.652350 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:14:09.652377 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 28 2011-11-03 11:14:09.652400 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.652416 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:14:09.652450 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 1 0x41ae5e50 client_session(open) v1 2011-11-03 11:14:09.652485 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.652518 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.652548 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41ae5e50 2011-11-03 11:14:09.652584 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 1 ==== client_session(open) v1 ==== 28+0+0 (2719320884 0 0) 0x41ae5e50 con 0x7fb27061cef0 2011-11-03 11:14:09.652603 7fb263e5c700 client.5563 handle_client_session client_session(open) v1 2011-11-03 11:14:09.652625 7fb263e5c700 client.5563 renew_caps mds.0 2011-11-03 11:14:09.652654 7fb263e5c700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_renewcaps seq 1) v1 -- ?+0 0x41ae61c0 2011-11-03 11:14:09.652691 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.652720 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 1 2011-11-03 11:14:09.652749 7fb263e5c700 -- 192.168.141.130:0/3011511 submit_message client_session(request_renewcaps seq 1) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.652774 7fb263e5c700 client.5563 connect_mds_targets for mds.0 2011-11-03 11:14:09.652802 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:14:09.652821 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41ae5e50 2011-11-03 11:14:09.652855 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 2 0x41ae61c0 client_session(request_renewcaps seq 1) v1 2011-11-03 11:14:09.652885 7fb269f70700 client.5563 send_request rebuilding request 1 for mds.0 2011-11-03 11:14:09.652912 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb270736000, mds: 0) 2011-11-03 11:14:09.652935 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.652928 2011-11-03 11:14:09.652968 7fb269f70700 client.5563 send_request client_request(unknown.0:1 getattr pAsLsXsFs #1) v1 to mds.0 2011-11-03 11:14:09.652994 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:1 getattr pAsLsXsFs #1) v1 -- ?+0 0x7fb270736610 2011-11-03 11:14:09.653021 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:1 getattr pAsLsXsFs #1) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.653040 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6e660 2011-11-03 11:14:09.653071 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 2 0x41ae61c0 2011-11-03 11:14:09.653094 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x41ae61c0 2011-11-03 11:14:09.653121 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.653150 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 3 0x7fb270736610 client_request(client.5563:1 getattr pAsLsXsFs #1) v1 2011-11-03 11:14:09.653171 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 3 0x7fb270736610 2011-11-03 11:14:09.653188 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270736610 2011-11-03 11:14:09.653230 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.653247 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.653392 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.653423 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.653445 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.653468 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:14:09.653485 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:14:09.653507 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 28 2011-11-03 11:14:09.653544 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.653560 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:14:09.653584 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 2 0x41ae5e50 client_session(renewcaps seq 1) v1 2011-11-03 11:14:09.653606 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.653631 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.653656 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41ae5e50 2011-11-03 11:14:09.653690 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 2 ==== client_session(renewcaps seq 1) v1 ==== 28+0+0 (2614397064 0 0) 0x41ae5e50 con 0x7fb27061cef0 2011-11-03 11:14:09.653709 7fb263e5c700 client.5563 handle_client_session client_session(renewcaps seq 1) v1 2011-11-03 11:14:09.653733 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:14:09.653752 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41ae5e50 2011-11-03 11:14:09.653784 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.653808 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 2 2011-11-03 11:14:09.653832 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.653852 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.653877 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.653909 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.653937 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.653974 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=316 data=0 off 0 2011-11-03 11:14:09.653991 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 316 from dispatch throttler 0/104857600 2011-11-03 11:14:09.654014 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 316 2011-11-03 11:14:09.654043 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.654060 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 316 + 0 + 0 byte message 2011-11-03 11:14:09.654097 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 3 0x41ae61c0 client_reply(???:1 = 0 Success) v1 2011-11-03 11:14:09.654122 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.654148 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.654173 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41ae61c0 2011-11-03 11:14:09.654208 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 3 ==== client_reply(???:1 = 0 Success) v1 ==== 316+0+0 (2486345212 0 0) 0x41ae61c0 con 0x7fb27061cef0 2011-11-03 11:14:09.654226 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 1 2011-11-03 11:14:09.654247 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.652928 mds.0 is_target=1 is_dentry=0 2011-11-03 11:14:09.654265 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.654284 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.654306 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=2 c=0 seq=0 parent=0 my_snaps=[] cached_snapc=0=[]) seq 1 > 0 2011-11-03 11:14:09.654325 7fb263e5c700 client.5563 flushing caps on snaprealm(1 nref=2 c=0 seq=0 parent=0 my_snaps=[] cached_snapc=0=[]) 2011-11-03 11:14:09.654346 7fb263e5c700 client.5563 invalidate_snaprealm_and_children snaprealm(1 nref=2 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=0=[]) 2011-11-03 11:14:09.654364 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=2 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=0=[]) self|parent updated 2011-11-03 11:14:09.654383 7fb263e5c700 client.5563 snapc 1=[] 2011-11-03 11:14:09.654402 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=0 2011-11-03 11:14:09.654432 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.654458 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 3 2011-11-03 11:14:09.654482 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.654503 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.654528 7fb263e5c700 inode.get on 0x41ae6510 1.head now 1 2011-11-03 11:14:09.654557 7fb263e5c700 client.5563 add_update_inode adding 1.head(ref=1 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41ae6510) caps pAsLsXsFs 2011-11-03 11:14:09.654570 7fb263e5c700 client.5563 dir hash is 2 2011-11-03 11:14:09.654588 7fb263e5c700 client.5563 update_inode_file_bits 1.head(ref=1 cap_refs={} open={} mode=40755 size=0 mtime=0.000000 caps=- 0x41ae6510) - mtime 2011-11-01 21:12:55.797267 2011-11-03 11:14:09.654620 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.654651 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXsFs from mds.0 on 1.head(ref=1 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) 2011-11-03 11:14:09.654666 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6e660 2011-11-03 11:14:09.654684 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 1 0x7fb263e5b490 2011-11-03 11:14:09.654713 7fb269f70700 client.5563 sendrecv kickback on tid 1 0x7fb263e5b490 2011-11-03 11:14:09.654734 7fb269f70700 client.5563 lat 0.001802 2011-11-03 11:14:09.654751 7fb269f70700 client.5563 root getattr result=0 2011-11-03 11:14:09.654766 7fb269f70700 inode.get on 0x41ae6510 1.head now 2 2011-11-03 11:14:09.654778 7fb269f70700 client.5563 _ll_get 0x41ae6510 1 -> 1 2011-11-03 11:14:09.654808 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 316 to dispatch throttler 316/104857600 2011-11-03 11:14:09.654827 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41ae61c0 2011-11-03 11:14:09.654959 7fb269f70700 client.5563 path_walk #1 2011-11-03 11:14:09.654978 7fb269f70700 client.5563 chdir(/) cwd now 1 2011-11-03 11:14:09.655162 7fb269f70700 client.5563 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:09.655187 7fb269f70700 client.5563 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:09.655216 7fb269f70700 client.5563 0 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp 2011-11-03 11:14:09.655242 7fb269f70700 client.5563 _lookup on #1/tmp 2011-11-03 11:14:09.655268 7fb269f70700 client.5563 choose_target_mds starting with req->inode 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) 2011-11-03 11:14:09.655284 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on tmp => 3207142759 2011-11-03 11:14:09.655306 7fb269f70700 client.5563 choose_target_mds 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) is_hash=1 hash=3207142759 2011-11-03 11:14:09.655328 7fb269f70700 client.5563 choose_target_mds from caps on inode 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) 2011-11-03 11:14:09.655340 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.655352 7fb269f70700 client.5563 send_request rebuilding request 2 for mds.0 2011-11-03 11:14:09.655365 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb270736000, mds: 0) 2011-11-03 11:14:09.655384 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.655379 2011-11-03 11:14:09.655398 7fb269f70700 client.5563 send_request client_request(unknown.0:2 lookup #1/tmp) v1 to mds.0 2011-11-03 11:14:09.655416 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:2 lookup #1/tmp) v1 -- ?+0 0x7fb270736610 2011-11-03 11:14:09.655435 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:2 lookup #1/tmp) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.655455 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6f0a0 2011-11-03 11:14:09.655488 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.655516 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 4 0x7fb270736610 client_request(client.5563:2 lookup #1/tmp) v1 2011-11-03 11:14:09.655546 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 4 0x7fb270736610 2011-11-03 11:14:09.655572 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270736610 2011-11-03 11:14:09.655599 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.655615 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.655838 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.655870 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.655893 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.655916 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=582 data=0 off 0 2011-11-03 11:14:09.655933 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 582 from dispatch throttler 0/104857600 2011-11-03 11:14:09.655961 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 582 2011-11-03 11:14:09.655983 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.656000 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 582 + 0 + 0 byte message 2011-11-03 11:14:09.656029 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 4 0x41cc2640 client_reply(???:2 = 0 Success) v1 2011-11-03 11:14:09.656054 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.656080 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.656105 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41cc2640 2011-11-03 11:14:09.656137 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 4 ==== client_reply(???:2 = 0 Success) v1 ==== 582+0+0 (702853116 0 0) 0x41cc2640 con 0x7fb27061cef0 2011-11-03 11:14:09.656156 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 2 2011-11-03 11:14:09.656176 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.655379 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.656193 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.656210 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.656231 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=3 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.656249 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.656281 7fb263e5c700 client.5563 add_update_inode adding 1000000005c.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41cc28b0) caps pAsLsXsFs 2011-11-03 11:14:09.656299 7fb263e5c700 client.5563 dir hash is 2 2011-11-03 11:14:09.656324 7fb263e5c700 client.5563 update_inode_file_bits 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x41cc28b0) - mtime 2011-11-01 21:06:06.505221 2011-11-03 11:14:09.656348 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.656390 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXsFs from mds.0 on 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc28b0) 2011-11-03 11:14:09.656420 7fb263e5c700 client.5563 add_update_inode had 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) caps pAsLsXsFs 2011-11-03 11:14:09.656453 7fb263e5c700 client.5563 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) 2011-11-03 11:14:09.656472 7fb263e5c700 client.5563 got dirfrag map for 1 frag * to mds 0 2011-11-03 11:14:09.656495 7fb263e5c700 client.5563 insert_dentry_inode 'tmp' vino 1000000005c.head in dir 1.head dn 0 2011-11-03 11:14:09.656513 7fb263e5c700 inode.get on 0x41cc28b0 1000000005c.head now 1 2011-11-03 11:14:09.656535 7fb263e5c700 client.5563 link dir 0x41ae6510 'tmp' to inode 0x41cc28b0 dn 0x41ae4050 (new dn) 2011-11-03 11:14:09.656553 7fb263e5c700 inode.get on 0x41cc28b0 1000000005c.head now 2 2011-11-03 11:14:09.656571 7fb263e5c700 client.5563 link inode 0x41cc28b0 parents now 0x41ae4050 2011-11-03 11:14:09.656590 7fb263e5c700 inode.put on 0x41cc28b0 1000000005c.head now 1 2011-11-03 11:14:09.656606 7fb263e5c700 client.5563 setting dn offset to 2 2011-11-03 11:14:09.656625 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6f0a0 2011-11-03 11:14:09.656646 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 2 0x7fb263e5b490 2011-11-03 11:14:09.656680 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.656707 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 4 2011-11-03 11:14:09.656732 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.656752 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.656776 7fb269f70700 client.5563 sendrecv kickback on tid 2 0x7fb263e5b490 2011-11-03 11:14:09.656799 7fb269f70700 client.5563 lat 0.001417 2011-11-03 11:14:09.656818 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.656853 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp = 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.656875 7fb269f70700 client.5563 1 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins 2011-11-03 11:14:09.656892 7fb269f70700 client.5563 _lookup on #1000000005c/hadoop-nwatkins 2011-11-03 11:14:09.656915 7fb269f70700 client.5563 choose_target_mds starting with req->inode 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.656929 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on hadoop-nwatkins => 2416329197 2011-11-03 11:14:09.656950 7fb269f70700 client.5563 choose_target_mds 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) is_hash=1 hash=2416329197 2011-11-03 11:14:09.656975 7fb269f70700 client.5563 choose_target_mds from caps on inode 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.657011 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.657025 7fb269f70700 client.5563 send_request rebuilding request 3 for mds.0 2011-11-03 11:14:09.657038 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb2707368e0, mds: 0) 2011-11-03 11:14:09.657053 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.657049 2011-11-03 11:14:09.657066 7fb269f70700 client.5563 send_request client_request(unknown.0:3 lookup #1000000005c/hadoop-nwatkins) v1 to mds.0 2011-11-03 11:14:09.657085 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:3 lookup #1000000005c/hadoop-nwatkins) v1 -- ?+0 0x7fb270736610 2011-11-03 11:14:09.657106 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:3 lookup #1000000005c/hadoop-nwatkins) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.657123 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6f0a0 2011-11-03 11:14:09.657159 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.657189 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 5 0x7fb270736610 client_request(client.5563:3 lookup #1000000005c/hadoop-nwatkins) v1 2011-11-03 11:14:09.657216 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 582 to dispatch throttler 582/104857600 2011-11-03 11:14:09.657241 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc2640 2011-11-03 11:14:09.657273 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 5 0x7fb270736610 2011-11-03 11:14:09.657295 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270736610 2011-11-03 11:14:09.657322 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.657338 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.657554 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.657585 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.657608 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.657631 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=594 data=0 off 0 2011-11-03 11:14:09.657648 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 594 from dispatch throttler 0/104857600 2011-11-03 11:14:09.657675 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 594 2011-11-03 11:14:09.657699 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.657715 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 594 + 0 + 0 byte message 2011-11-03 11:14:09.657743 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 5 0x41cc2640 client_reply(???:3 = 0 Success) v1 2011-11-03 11:14:09.657767 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.657807 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.657839 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41cc2640 2011-11-03 11:14:09.657870 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 5 ==== client_reply(???:3 = 0 Success) v1 ==== 594+0+0 (2582640975 0 0) 0x41cc2640 con 0x7fb27061cef0 2011-11-03 11:14:09.657888 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 3 2011-11-03 11:14:09.657908 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.657049 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.657925 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.657942 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.657963 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=4 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.657980 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.658009 7fb263e5c700 client.5563 add_update_inode adding 1000000005d.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41cc3300) caps pAsLsXsFs 2011-11-03 11:14:09.658027 7fb263e5c700 client.5563 dir hash is 2 2011-11-03 11:14:09.658047 7fb263e5c700 client.5563 update_inode_file_bits 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x41cc3300) - mtime 2011-11-01 21:06:06.505963 2011-11-03 11:14:09.658066 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.658099 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXsFs from mds.0 on 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc3300) 2011-11-03 11:14:09.658130 7fb263e5c700 client.5563 add_update_inode had 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) caps pAsLsXsFs 2011-11-03 11:14:09.658163 7fb263e5c700 client.5563 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.658182 7fb263e5c700 client.5563 got dirfrag map for 1000000005c frag * to mds -1 2011-11-03 11:14:09.658204 7fb263e5c700 client.5563 insert_dentry_inode 'hadoop-nwatkins' vino 1000000005d.head in dir 1000000005c.head dn 0 2011-11-03 11:14:09.658221 7fb263e5c700 inode.get on 0x41cc3300 1000000005d.head now 1 2011-11-03 11:14:09.658240 7fb263e5c700 client.5563 link dir 0x41cc28b0 'hadoop-nwatkins' to inode 0x41cc3300 dn 0x41ae5e50 (new dn) 2011-11-03 11:14:09.658260 7fb263e5c700 inode.get on 0x41cc3300 1000000005d.head now 2 2011-11-03 11:14:09.658279 7fb263e5c700 client.5563 link inode 0x41cc3300 parents now 0x41ae5e50 2011-11-03 11:14:09.658296 7fb263e5c700 inode.put on 0x41cc3300 1000000005d.head now 1 2011-11-03 11:14:09.658313 7fb263e5c700 client.5563 setting dn offset to 2 2011-11-03 11:14:09.658331 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6f0a0 2011-11-03 11:14:09.658352 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 3 0x7fb263e5b490 2011-11-03 11:14:09.658388 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.658413 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 5 2011-11-03 11:14:09.658438 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.658469 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.658506 7fb269f70700 client.5563 sendrecv kickback on tid 3 0x7fb263e5b490 2011-11-03 11:14:09.658529 7fb269f70700 client.5563 lat 0.001476 2011-11-03 11:14:09.658547 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.658580 7fb269f70700 client.5563 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins = 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.658603 7fb269f70700 client.5563 2 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred 2011-11-03 11:14:09.658620 7fb269f70700 client.5563 _lookup on #1000000005d/mapred 2011-11-03 11:14:09.658642 7fb269f70700 client.5563 choose_target_mds starting with req->inode 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.658655 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on mapred => 1878159916 2011-11-03 11:14:09.658680 7fb269f70700 client.5563 choose_target_mds 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) is_hash=1 hash=1878159916 2011-11-03 11:14:09.658702 7fb269f70700 client.5563 choose_target_mds from caps on inode 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.658715 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.658727 7fb269f70700 client.5563 send_request rebuilding request 4 for mds.0 2011-11-03 11:14:09.658740 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb270736000, mds: 0) 2011-11-03 11:14:09.658755 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.658751 2011-11-03 11:14:09.658768 7fb269f70700 client.5563 send_request client_request(unknown.0:4 lookup #1000000005d/mapred) v1 to mds.0 2011-11-03 11:14:09.658786 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:4 lookup #1000000005d/mapred) v1 -- ?+0 0x7fb270736610 2011-11-03 11:14:09.658805 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:4 lookup #1000000005d/mapred) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.658825 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6f0a0 2011-11-03 11:14:09.658860 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.658889 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 6 0x7fb270736610 client_request(client.5563:4 lookup #1000000005d/mapred) v1 2011-11-03 11:14:09.658915 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 6 0x7fb270736610 2011-11-03 11:14:09.658936 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270736610 2011-11-03 11:14:09.658963 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 594 to dispatch throttler 594/104857600 2011-11-03 11:14:09.658987 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc2640 2011-11-03 11:14:09.659019 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.659047 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.659274 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.659305 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.659327 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.659350 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=585 data=0 off 0 2011-11-03 11:14:09.659367 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 585 from dispatch throttler 0/104857600 2011-11-03 11:14:09.659395 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 585 2011-11-03 11:14:09.659418 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.659435 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 585 + 0 + 0 byte message 2011-11-03 11:14:09.659461 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 6 0x41cc2640 client_reply(???:4 = 0 Success) v1 2011-11-03 11:14:09.659485 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.659511 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.659537 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41cc2640 2011-11-03 11:14:09.659567 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 6 ==== client_reply(???:4 = 0 Success) v1 ==== 585+0+0 (2417356543 0 0) 0x41cc2640 con 0x7fb27061cef0 2011-11-03 11:14:09.659586 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 4 2011-11-03 11:14:09.659606 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.658751 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.659622 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.659639 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.659660 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=5 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.659678 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.659706 7fb263e5c700 client.5563 add_update_inode adding 1000000005e.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41cc6090) caps pAsLsXs 2011-11-03 11:14:09.659724 7fb263e5c700 client.5563 dir hash is 2 2011-11-03 11:14:09.659744 7fb263e5c700 client.5563 update_inode_file_bits 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x41cc6090) - mtime 2011-11-03 11:13:37.993864 2011-11-03 11:14:09.659763 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.659795 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXs from mds.0 on 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) 0x41cc6090) 2011-11-03 11:14:09.659827 7fb263e5c700 client.5563 add_update_inode had 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) caps pAsLsXsFs 2011-11-03 11:14:09.659866 7fb263e5c700 client.5563 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.659893 7fb263e5c700 client.5563 got dirfrag map for 1000000005d frag * to mds -1 2011-11-03 11:14:09.659916 7fb263e5c700 client.5563 insert_dentry_inode 'mapred' vino 1000000005e.head in dir 1000000005d.head dn 0 2011-11-03 11:14:09.659934 7fb263e5c700 inode.get on 0x41cc6090 1000000005e.head now 1 2011-11-03 11:14:09.659952 7fb263e5c700 client.5563 link dir 0x41cc3300 'mapred' to inode 0x41cc6090 dn 0x41cc3eb0 (new dn) 2011-11-03 11:14:09.659970 7fb263e5c700 inode.get on 0x41cc6090 1000000005e.head now 2 2011-11-03 11:14:09.659988 7fb263e5c700 client.5563 link inode 0x41cc6090 parents now 0x41cc3eb0 2011-11-03 11:14:09.660005 7fb263e5c700 inode.put on 0x41cc6090 1000000005e.head now 1 2011-11-03 11:14:09.660021 7fb263e5c700 client.5563 setting dn offset to 2 2011-11-03 11:14:09.660039 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6f0a0 2011-11-03 11:14:09.660060 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 4 0x7fb263e5b490 2011-11-03 11:14:09.660092 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.660117 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 6 2011-11-03 11:14:09.660141 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.660161 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.660183 7fb269f70700 client.5563 sendrecv kickback on tid 4 0x7fb263e5b490 2011-11-03 11:14:09.660204 7fb269f70700 client.5563 lat 0.001450 2011-11-03 11:14:09.660218 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.660253 7fb269f70700 client.5563 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred = 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.660275 7fb269f70700 client.5563 3 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging 2011-11-03 11:14:09.660293 7fb269f70700 client.5563 _lookup on #1000000005e/staging 2011-11-03 11:14:09.660314 7fb269f70700 client.5563 choose_target_mds starting with req->inode 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.660328 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on staging => 875688800 2011-11-03 11:14:09.660348 7fb269f70700 client.5563 choose_target_mds 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) is_hash=1 hash=875688800 2011-11-03 11:14:09.660369 7fb269f70700 client.5563 choose_target_mds from caps on inode 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.660381 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.660396 7fb269f70700 client.5563 send_request rebuilding request 5 for mds.0 2011-11-03 11:14:09.660409 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb2707368e0, mds: 0) 2011-11-03 11:14:09.660424 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.660420 2011-11-03 11:14:09.660437 7fb269f70700 client.5563 send_request client_request(unknown.0:5 lookup #1000000005e/staging) v1 to mds.0 2011-11-03 11:14:09.660476 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:5 lookup #1000000005e/staging) v1 -- ?+0 0x7fb270736610 2011-11-03 11:14:09.660494 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:5 lookup #1000000005e/staging) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.660511 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6f0a0 2011-11-03 11:14:09.660547 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.660576 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 7 0x7fb270736610 client_request(client.5563:5 lookup #1000000005e/staging) v1 2011-11-03 11:14:09.660601 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 7 0x7fb270736610 2011-11-03 11:14:09.660623 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270736610 2011-11-03 11:14:09.660648 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 585 to dispatch throttler 585/104857600 2011-11-03 11:14:09.660671 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc2640 2011-11-03 11:14:09.660704 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.660726 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.660994 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.661024 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.661046 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.661068 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=586 data=0 off 0 2011-11-03 11:14:09.661085 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 586 from dispatch throttler 0/104857600 2011-11-03 11:14:09.661109 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 586 2011-11-03 11:14:09.661135 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.661151 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 586 + 0 + 0 byte message 2011-11-03 11:14:09.661178 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 7 0x41cc6c30 client_reply(???:5 = 0 Success) v1 2011-11-03 11:14:09.661202 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.661227 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.661253 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41cc6c30 2011-11-03 11:14:09.661282 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 7 ==== client_reply(???:5 = 0 Success) v1 ==== 586+0+0 (2147591670 0 0) 0x41cc6c30 con 0x7fb27061cef0 2011-11-03 11:14:09.661315 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 5 2011-11-03 11:14:09.661335 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.660420 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.661351 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.661368 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.661388 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=6 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.661405 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.661435 7fb263e5c700 client.5563 add_update_inode adding 100000008ad.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41cc6ea0) caps pAsLsXsFs 2011-11-03 11:14:09.661452 7fb263e5c700 client.5563 dir hash is 2 2011-11-03 11:14:09.661473 7fb263e5c700 client.5563 update_inode_file_bits 100000008ad.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=0.000000 caps=- 0x41cc6ea0) - mtime 2011-11-01 21:07:33.717434 2011-11-03 11:14:09.661491 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.661523 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXsFs from mds.0 on 100000008ad.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc6ea0) 2011-11-03 11:14:09.661555 7fb263e5c700 client.5563 add_update_inode had 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) caps pAsLsXs 2011-11-03 11:14:09.661587 7fb263e5c700 client.5563 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.661605 7fb263e5c700 client.5563 got dirfrag map for 1000000005e frag * to mds -1 2011-11-03 11:14:09.661626 7fb263e5c700 client.5563 insert_dentry_inode 'staging' vino 100000008ad.head in dir 1000000005e.head dn 0 2011-11-03 11:14:09.661644 7fb263e5c700 inode.get on 0x41cc6ea0 100000008ad.head now 1 2011-11-03 11:14:09.661662 7fb263e5c700 client.5563 link dir 0x41cc6090 'staging' to inode 0x41cc6ea0 dn 0x41ae61f0 (new dn) 2011-11-03 11:14:09.661679 7fb263e5c700 inode.get on 0x41cc6ea0 100000008ad.head now 2 2011-11-03 11:14:09.661696 7fb263e5c700 client.5563 link inode 0x41cc6ea0 parents now 0x41ae61f0 2011-11-03 11:14:09.661713 7fb263e5c700 inode.put on 0x41cc6ea0 100000008ad.head now 1 2011-11-03 11:14:09.661729 7fb263e5c700 client.5563 setting dn offset to 2 2011-11-03 11:14:09.661744 7fb263e5c700 client.5563 got dentry lease on staging dur 30000ms ttl 2011-11-03 11:14:39.660420 2011-11-03 11:14:09.661761 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6f0a0 2011-11-03 11:14:09.661783 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 5 0x7fb263e5b490 2011-11-03 11:14:09.661814 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.661840 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 7 2011-11-03 11:14:09.661864 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.661884 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.661906 7fb269f70700 client.5563 sendrecv kickback on tid 5 0x7fb263e5b490 2011-11-03 11:14:09.661926 7fb269f70700 client.5563 lat 0.001503 2011-11-03 11:14:09.661940 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.661975 7fb269f70700 client.5563 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging = 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.662012 7fb269f70700 client.5563 4 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins 2011-11-03 11:14:09.662029 7fb269f70700 client.5563 _lookup on #100000008ad/nwatkins 2011-11-03 11:14:09.662052 7fb269f70700 client.5563 choose_target_mds starting with req->inode 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.662065 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on nwatkins => 2332913853 2011-11-03 11:14:09.662086 7fb269f70700 client.5563 choose_target_mds 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) is_hash=1 hash=2332913853 2011-11-03 11:14:09.662108 7fb269f70700 client.5563 choose_target_mds from caps on inode 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.662123 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.662135 7fb269f70700 client.5563 send_request rebuilding request 6 for mds.0 2011-11-03 11:14:09.662148 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb270736000, mds: 0) 2011-11-03 11:14:09.662163 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.662159 2011-11-03 11:14:09.662177 7fb269f70700 client.5563 send_request client_request(unknown.0:6 lookup #100000008ad/nwatkins) v1 to mds.0 2011-11-03 11:14:09.662195 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:6 lookup #100000008ad/nwatkins) v1 -- ?+0 0x7fb270736610 2011-11-03 11:14:09.662213 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:6 lookup #100000008ad/nwatkins) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.662229 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6f0a0 2011-11-03 11:14:09.662265 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.662295 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 8 0x7fb270736610 client_request(client.5563:6 lookup #100000008ad/nwatkins) v1 2011-11-03 11:14:09.662320 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 8 0x7fb270736610 2011-11-03 11:14:09.662341 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270736610 2011-11-03 11:14:09.662367 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 586 to dispatch throttler 586/104857600 2011-11-03 11:14:09.662391 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc6c30 2011-11-03 11:14:09.662424 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.662446 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.662715 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.662745 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.662773 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.662806 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=587 data=0 off 0 2011-11-03 11:14:09.662824 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 587 from dispatch throttler 0/104857600 2011-11-03 11:14:09.662851 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 587 2011-11-03 11:14:09.662874 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.662890 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 587 + 0 + 0 byte message 2011-11-03 11:14:09.662917 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 8 0x41cc6c30 client_reply(???:6 = 0 Success) v1 2011-11-03 11:14:09.662941 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.662967 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.662999 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41cc6c30 2011-11-03 11:14:09.663029 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 8 ==== client_reply(???:6 = 0 Success) v1 ==== 587+0+0 (3588036753 0 0) 0x41cc6c30 con 0x7fb27061cef0 2011-11-03 11:14:09.663048 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 6 2011-11-03 11:14:09.663068 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.662159 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.663085 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.663102 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.663123 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=7 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.663140 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.663169 7fb263e5c700 client.5563 add_update_inode adding 100000008ae.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41cc7970) caps pAsLsXsFs 2011-11-03 11:14:09.663187 7fb263e5c700 client.5563 dir hash is 2 2011-11-03 11:14:09.663208 7fb263e5c700 client.5563 update_inode_file_bits 100000008ae.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=0.000000 caps=- 0x41cc7970) - mtime 2011-11-01 21:07:33.718516 2011-11-03 11:14:09.663227 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.663260 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXsFs from mds.0 on 100000008ae.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc7970) 2011-11-03 11:14:09.663289 7fb263e5c700 client.5563 add_update_inode had 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) caps pAsLsXsFs 2011-11-03 11:14:09.663323 7fb263e5c700 client.5563 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.663342 7fb263e5c700 client.5563 got dirfrag map for 100000008ad frag * to mds -1 2011-11-03 11:14:09.663362 7fb263e5c700 client.5563 insert_dentry_inode 'nwatkins' vino 100000008ae.head in dir 100000008ad.head dn 0 2011-11-03 11:14:09.663387 7fb263e5c700 inode.get on 0x41cc7970 100000008ae.head now 1 2011-11-03 11:14:09.663418 7fb263e5c700 client.5563 link dir 0x41cc6ea0 'nwatkins' to inode 0x41cc7970 dn 0x41cc7ee0 (new dn) 2011-11-03 11:14:09.663436 7fb263e5c700 inode.get on 0x41cc7970 100000008ae.head now 2 2011-11-03 11:14:09.663452 7fb263e5c700 client.5563 link inode 0x41cc7970 parents now 0x41cc7ee0 2011-11-03 11:14:09.663469 7fb263e5c700 inode.put on 0x41cc7970 100000008ae.head now 1 2011-11-03 11:14:09.663486 7fb263e5c700 client.5563 setting dn offset to 2 2011-11-03 11:14:09.663505 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6f0a0 2011-11-03 11:14:09.663527 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 6 0x7fb263e5b490 2011-11-03 11:14:09.663562 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.663588 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 8 2011-11-03 11:14:09.663612 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.663631 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.663654 7fb269f70700 client.5563 sendrecv kickback on tid 6 0x7fb263e5b490 2011-11-03 11:14:09.663674 7fb269f70700 client.5563 lat 0.001512 2011-11-03 11:14:09.663688 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.663725 7fb269f70700 client.5563 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins = 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.663747 7fb269f70700 client.5563 5 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging 2011-11-03 11:14:09.663764 7fb269f70700 client.5563 _lookup on #100000008ae/.staging 2011-11-03 11:14:09.663785 7fb269f70700 client.5563 choose_target_mds starting with req->inode 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.663799 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on .staging => 1933209695 2011-11-03 11:14:09.663820 7fb269f70700 client.5563 choose_target_mds 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) is_hash=1 hash=1933209695 2011-11-03 11:14:09.663845 7fb269f70700 client.5563 choose_target_mds from caps on inode 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.663857 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.663869 7fb269f70700 client.5563 send_request rebuilding request 7 for mds.0 2011-11-03 11:14:09.663882 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb2707368e0, mds: 0) 2011-11-03 11:14:09.663897 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.663893 2011-11-03 11:14:09.663911 7fb269f70700 client.5563 send_request client_request(unknown.0:7 lookup #100000008ae/.staging) v1 to mds.0 2011-11-03 11:14:09.663928 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:7 lookup #100000008ae/.staging) v1 -- ?+0 0x7fb270736610 2011-11-03 11:14:09.663947 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:7 lookup #100000008ae/.staging) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.663969 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6f0a0 2011-11-03 11:14:09.664015 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.664045 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 9 0x7fb270736610 client_request(client.5563:7 lookup #100000008ae/.staging) v1 2011-11-03 11:14:09.664069 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 9 0x7fb270736610 2011-11-03 11:14:09.664091 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270736610 2011-11-03 11:14:09.664116 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 587 to dispatch throttler 587/104857600 2011-11-03 11:14:09.664140 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc6c30 2011-11-03 11:14:09.664172 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.664194 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.664434 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.664464 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.664487 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.664510 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=587 data=0 off 0 2011-11-03 11:14:09.664527 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 587 from dispatch throttler 0/104857600 2011-11-03 11:14:09.664554 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 587 2011-11-03 11:14:09.664576 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.664593 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 587 + 0 + 0 byte message 2011-11-03 11:14:09.664620 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 9 0x41cc4660 client_reply(???:7 = 0 Success) v1 2011-11-03 11:14:09.664642 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.664668 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.664694 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41cc4660 2011-11-03 11:14:09.664723 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 9 ==== client_reply(???:7 = 0 Success) v1 ==== 587+0+0 (2757524062 0 0) 0x41cc4660 con 0x7fb27061cef0 2011-11-03 11:14:09.664742 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 7 2011-11-03 11:14:09.664760 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.663893 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.664776 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.664793 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.664820 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=8 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.664845 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.664875 7fb263e5c700 client.5563 add_update_inode adding 100000008af.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41cc48d0) caps pAsLsXsFs 2011-11-03 11:14:09.664893 7fb263e5c700 client.5563 dir hash is 2 2011-11-03 11:14:09.664913 7fb263e5c700 client.5563 update_inode_file_bits 100000008af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=0.000000 caps=- 0x41cc48d0) - mtime 2011-11-03 11:14:02.107013 2011-11-03 11:14:09.664932 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.664965 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXsFs from mds.0 on 100000008af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc48d0) 2011-11-03 11:14:09.664997 7fb263e5c700 client.5563 add_update_inode had 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) caps pAsLsXsFs 2011-11-03 11:14:09.665030 7fb263e5c700 client.5563 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.665049 7fb263e5c700 client.5563 got dirfrag map for 100000008ae frag * to mds -1 2011-11-03 11:14:09.665069 7fb263e5c700 client.5563 insert_dentry_inode '.staging' vino 100000008af.head in dir 100000008ae.head dn 0 2011-11-03 11:14:09.665088 7fb263e5c700 inode.get on 0x41cc48d0 100000008af.head now 1 2011-11-03 11:14:09.665106 7fb263e5c700 client.5563 link dir 0x41cc7970 '.staging' to inode 0x41cc48d0 dn 0x41cc2670 (new dn) 2011-11-03 11:14:09.665123 7fb263e5c700 inode.get on 0x41cc48d0 100000008af.head now 2 2011-11-03 11:14:09.665141 7fb263e5c700 client.5563 link inode 0x41cc48d0 parents now 0x41cc2670 2011-11-03 11:14:09.665158 7fb263e5c700 inode.put on 0x41cc48d0 100000008af.head now 1 2011-11-03 11:14:09.665175 7fb263e5c700 client.5563 setting dn offset to 2 2011-11-03 11:14:09.665194 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6f0a0 2011-11-03 11:14:09.665216 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 7 0x7fb263e5b490 2011-11-03 11:14:09.665251 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.665277 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 9 2011-11-03 11:14:09.665301 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.665321 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.665343 7fb269f70700 client.5563 sendrecv kickback on tid 7 0x7fb263e5b490 2011-11-03 11:14:09.665363 7fb269f70700 client.5563 lat 0.001467 2011-11-03 11:14:09.665377 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.665410 7fb269f70700 client.5563 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging = 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.665436 7fb269f70700 client.5563 6 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 2011-11-03 11:14:09.665460 7fb269f70700 client.5563 _lookup on #100000008af/job_201111031113_0001 2011-11-03 11:14:09.665491 7fb269f70700 client.5563 choose_target_mds starting with req->inode 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.665504 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on job_201111031113_0001 => 3184690367 2011-11-03 11:14:09.665525 7fb269f70700 client.5563 choose_target_mds 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) is_hash=1 hash=3184690367 2011-11-03 11:14:09.665546 7fb269f70700 client.5563 choose_target_mds from caps on inode 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.665561 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.665574 7fb269f70700 client.5563 send_request rebuilding request 8 for mds.0 2011-11-03 11:14:09.665587 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb270736000, mds: 0) 2011-11-03 11:14:09.665602 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.665598 2011-11-03 11:14:09.665615 7fb269f70700 client.5563 send_request client_request(unknown.0:8 lookup #100000008af/job_201111031113_0001) v1 to mds.0 2011-11-03 11:14:09.665633 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:8 lookup #100000008af/job_201111031113_0001) v1 -- ?+0 0x7fb270736610 2011-11-03 11:14:09.665652 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:8 lookup #100000008af/job_201111031113_0001) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.665669 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6f0a0 2011-11-03 11:14:09.665704 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.665733 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 10 0x7fb270736610 client_request(client.5563:8 lookup #100000008af/job_201111031113_0001) v1 2011-11-03 11:14:09.665758 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 10 0x7fb270736610 2011-11-03 11:14:09.665780 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270736610 2011-11-03 11:14:09.665805 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 587 to dispatch throttler 587/104857600 2011-11-03 11:14:09.665828 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc4660 2011-11-03 11:14:09.665862 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.665884 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.666011 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.666041 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.666063 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.666087 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=600 data=0 off 0 2011-11-03 11:14:09.666117 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 600 from dispatch throttler 0/104857600 2011-11-03 11:14:09.666155 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 600 2011-11-03 11:14:09.666177 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.666194 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 600 + 0 + 0 byte message 2011-11-03 11:14:09.666220 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 10 0x41cc4660 client_reply(???:8 = 0 Success) v1 2011-11-03 11:14:09.666244 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.666270 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.666300 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41cc4660 2011-11-03 11:14:09.666330 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 10 ==== client_reply(???:8 = 0 Success) v1 ==== 600+0+0 (1521791972 0 0) 0x41cc4660 con 0x7fb27061cef0 2011-11-03 11:14:09.666349 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 8 2011-11-03 11:14:09.666369 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.665598 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.666385 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.666402 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.666421 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=9 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.666439 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.666466 7fb263e5c700 client.5563 add_update_inode adding 10000000982.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41cc53e0) caps pAsLsXs 2011-11-03 11:14:09.666484 7fb263e5c700 client.5563 dir hash is 2 2011-11-03 11:14:09.666504 7fb263e5c700 client.5563 update_inode_file_bits 10000000982.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=0.000000 caps=- 0x41cc53e0) - mtime 2011-11-03 11:14:04.597264 2011-11-03 11:14:09.666523 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.666555 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXs from mds.0 on 10000000982.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) 0x41cc53e0) 2011-11-03 11:14:09.666584 7fb263e5c700 client.5563 add_update_inode had 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) caps pAsLsXsFs 2011-11-03 11:14:09.666618 7fb263e5c700 client.5563 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.666637 7fb263e5c700 client.5563 got dirfrag map for 100000008af frag * to mds -1 2011-11-03 11:14:09.666659 7fb263e5c700 client.5563 insert_dentry_inode 'job_201111031113_0001' vino 10000000982.head in dir 100000008af.head dn 0 2011-11-03 11:14:09.666676 7fb263e5c700 inode.get on 0x41cc53e0 10000000982.head now 1 2011-11-03 11:14:09.666695 7fb263e5c700 client.5563 link dir 0x41cc48d0 'job_201111031113_0001' to inode 0x41cc53e0 dn 0x41cc5920 (new dn) 2011-11-03 11:14:09.666713 7fb263e5c700 inode.get on 0x41cc53e0 10000000982.head now 2 2011-11-03 11:14:09.666730 7fb263e5c700 client.5563 link inode 0x41cc53e0 parents now 0x41cc5920 2011-11-03 11:14:09.666754 7fb263e5c700 inode.put on 0x41cc53e0 10000000982.head now 1 2011-11-03 11:14:09.666782 7fb263e5c700 client.5563 setting dn offset to 2 2011-11-03 11:14:09.666802 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6f0a0 2011-11-03 11:14:09.666824 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 8 0x7fb263e5b490 2011-11-03 11:14:09.666860 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.666886 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 10 2011-11-03 11:14:09.666910 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.666930 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.666952 7fb269f70700 client.5563 sendrecv kickback on tid 8 0x7fb263e5b490 2011-11-03 11:14:09.666972 7fb269f70700 client.5563 lat 0.001371 2011-11-03 11:14:09.666986 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.667022 7fb269f70700 client.5563 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 = 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.667044 7fb269f70700 client.5563 7 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml 2011-11-03 11:14:09.667062 7fb269f70700 client.5563 _lookup on #10000000982/job.xml 2011-11-03 11:14:09.667084 7fb269f70700 client.5563 choose_target_mds starting with req->inode 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.667097 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on job.xml => 3152429972 2011-11-03 11:14:09.667118 7fb269f70700 client.5563 choose_target_mds 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) is_hash=1 hash=3152429972 2011-11-03 11:14:09.667142 7fb269f70700 client.5563 choose_target_mds from caps on inode 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.667155 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.667167 7fb269f70700 client.5563 send_request rebuilding request 9 for mds.0 2011-11-03 11:14:09.667179 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb2707368e0, mds: 0) 2011-11-03 11:14:09.667194 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.667190 2011-11-03 11:14:09.667207 7fb269f70700 client.5563 send_request client_request(unknown.0:9 lookup #10000000982/job.xml) v1 to mds.0 2011-11-03 11:14:09.667225 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:9 lookup #10000000982/job.xml) v1 -- ?+0 0x7fb270736610 2011-11-03 11:14:09.667244 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:9 lookup #10000000982/job.xml) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.667260 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6f0a0 2011-11-03 11:14:09.667292 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.667322 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 11 0x7fb270736610 client_request(client.5563:9 lookup #10000000982/job.xml) v1 2011-11-03 11:14:09.667361 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 11 0x7fb270736610 2011-11-03 11:14:09.667383 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270736610 2011-11-03 11:14:09.667408 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 600 to dispatch throttler 600/104857600 2011-11-03 11:14:09.667432 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc4660 2011-11-03 11:14:09.667464 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.667486 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.667731 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.667762 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.667785 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.667808 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=586 data=0 off 0 2011-11-03 11:14:09.667825 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 586 from dispatch throttler 0/104857600 2011-11-03 11:14:09.667852 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 586 2011-11-03 11:14:09.667876 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.667892 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 586 + 0 + 0 byte message 2011-11-03 11:14:09.667919 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 11 0x41cc5a20 client_reply(???:9 = 0 Success) v1 2011-11-03 11:14:09.667943 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.667970 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.667995 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41cc5a20 2011-11-03 11:14:09.668025 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 11 ==== client_reply(???:9 = 0 Success) v1 ==== 586+0+0 (2960170318 0 0) 0x41cc5a20 con 0x7fb27061cef0 2011-11-03 11:14:09.668043 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 9 2011-11-03 11:14:09.668063 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.667190 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.668080 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.668097 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.668117 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=10 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.668136 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.668166 7fb263e5c700 client.5563 add_update_inode adding 10000000986.head(ref=0 cap_refs={} open={} mode=100000 size=0 mtime=0.000000 caps=- objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] 0x41bc45b0) caps pAsLsXsFscr 2011-11-03 11:14:09.668207 7fb263e5c700 client.5563 update_inode_file_bits 10000000986.head(ref=0 cap_refs={} open={} mode=100644 size=0 mtime=0.000000 caps=- objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] 0x41bc45b0) - mtime 2011-11-03 11:14:04.597264 2011-11-03 11:14:09.668225 7fb263e5c700 client.5563 size 0 -> 17110 2011-11-03 11:14:09.668242 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.668283 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXsFscr from mds.0 on 10000000986.head(ref=0 cap_refs={} open={} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] 0x41bc45b0) 2011-11-03 11:14:09.668314 7fb263e5c700 client.5563 add_update_inode had 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) caps pAsLsXs 2011-11-03 11:14:09.668347 7fb263e5c700 client.5563 add_update_cap issued pAsLsXs -> pAsLsXs from mds.0 on 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.668365 7fb263e5c700 client.5563 got dirfrag map for 10000000982 frag * to mds -1 2011-11-03 11:14:09.668385 7fb263e5c700 client.5563 insert_dentry_inode 'job.xml' vino 10000000986.head in dir 10000000982.head dn 0 2011-11-03 11:14:09.668403 7fb263e5c700 inode.get on 0x41bc45b0 10000000986.head now 1 2011-11-03 11:14:09.668423 7fb263e5c700 client.5563 link dir 0x41cc53e0 'job.xml' to inode 0x41bc45b0 dn 0x41cc6c30 (new dn) 2011-11-03 11:14:09.668441 7fb263e5c700 inode.get on 0x41bc45b0 10000000986.head now 2 2011-11-03 11:14:09.668459 7fb263e5c700 client.5563 link inode 0x41bc45b0 parents now 0x41cc6c30 2011-11-03 11:14:09.668476 7fb263e5c700 inode.put on 0x41bc45b0 10000000986.head now 1 2011-11-03 11:14:09.668493 7fb263e5c700 client.5563 setting dn offset to 2 2011-11-03 11:14:09.668511 7fb263e5c700 client.5563 got dentry lease on job.xml dur 30000ms ttl 2011-11-03 11:14:39.667190 2011-11-03 11:14:09.668530 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6f0a0 2011-11-03 11:14:09.668551 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 9 0x7fb263e5b490 2011-11-03 11:14:09.668585 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.668611 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 11 2011-11-03 11:14:09.668635 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.668656 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.668677 7fb269f70700 client.5563 sendrecv kickback on tid 9 0x7fb263e5b490 2011-11-03 11:14:09.668698 7fb269f70700 client.5563 lat 0.001505 2011-11-03 11:14:09.668715 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.668750 7fb269f70700 client.5563 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.668770 7fb269f70700 client.5563 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.668787 7fb269f70700 client.5563 fill_stat on 10000000986 snap/devhead mode 0100644 mtime 2011-11-03 11:14:04.597264 ctime 2011-11-03 11:14:04.597264 2011-11-03 11:14:09.668809 7fb269f70700 client.5563 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:09.668846 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 586 to dispatch throttler 586/104857600 2011-11-03 11:14:09.668870 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc5a20 2011-11-03 11:14:09.668892 7fb269f70700 client.5563 open enter(/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0,0) = 2011-11-03 11:14:09.668916 7fb269f70700 client.5563 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:09.668939 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp 2011-11-03 11:14:09.668954 7fb269f70700 client.5563 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.668989 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.669015 7fb269f70700 client.5563 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins 2011-11-03 11:14:09.669029 7fb269f70700 client.5563 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.669063 7fb269f70700 client.5563 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.669085 7fb269f70700 client.5563 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred 2011-11-03 11:14:09.669099 7fb269f70700 client.5563 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.669133 7fb269f70700 client.5563 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.669157 7fb269f70700 client.5563 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging 2011-11-03 11:14:09.669172 7fb269f70700 client.5563 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:39.660420 seq 2 2011-11-03 11:14:09.669205 7fb269f70700 client.5563 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.669227 7fb269f70700 client.5563 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins 2011-11-03 11:14:09.669240 7fb269f70700 client.5563 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.669274 7fb269f70700 client.5563 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.669306 7fb269f70700 client.5563 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging 2011-11-03 11:14:09.669328 7fb269f70700 client.5563 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.669363 7fb269f70700 client.5563 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.669385 7fb269f70700 client.5563 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 2011-11-03 11:14:09.669399 7fb269f70700 client.5563 _lookup have dn job_201111031113_0001 mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.669435 7fb269f70700 client.5563 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.669456 7fb269f70700 client.5563 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml 2011-11-03 11:14:09.669471 7fb269f70700 client.5563 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:39.667190 seq 3 2011-11-03 11:14:09.669505 7fb269f70700 client.5563 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.669539 7fb269f70700 client.5563 check_caps on 10000000986.head(ref=1 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) wanted pFscr used - is_delayed=1 2011-11-03 11:14:09.669559 7fb269f70700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.669596 7fb269f70700 client.5563 send_cap 10000000986.head(ref=1 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) mds.0 seq 1 used - want pFscr flush - retain pAsxLsxXsxFsxcrwbl held pAsLsXsFscr revoking - dropping - 2011-11-03 11:14:09.669611 7fb269f70700 client.5563 auth cap, setting max_size = 0 2011-11-03 11:14:09.669639 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 10000000986 6774 seq 1 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 -- ?+0 0x7fb270736be0 2011-11-03 11:14:09.669663 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_caps(update ino 10000000986 6774 seq 1 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.669682 7fb269f70700 inode.get on 0x41bc45b0 10000000986.head now 2 2011-11-03 11:14:09.669698 7fb269f70700 client.5563 _create_fh 10000000986 mode 1 2011-11-03 11:14:09.669720 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.669762 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 12 0x7fb270736be0 client_caps(update ino 10000000986 6774 seq 1 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 2011-11-03 11:14:09.669801 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 12 0x7fb270736be0 2011-11-03 11:14:09.669823 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270736be0 2011-11-03 11:14:09.669845 7fb269f70700 client.5563 trim_cache size 8 max 16384 2011-11-03 11:14:09.669880 7fb269f70700 client.5563 open exit(#1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0) = 10 2011-11-03 11:14:09.669900 7fb269f70700 client.5563 describe_layout(10) = 0 2011-11-03 11:14:09.669918 7fb269f70700 client.5563 close enter(10) 2011-11-03 11:14:09.669949 7fb269f70700 client.5563 _release_fh 0x7fb270631920 mode 1 on 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.669980 7fb269f70700 client.5563 _flush 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.669996 7fb269f70700 client.5563 nothing to flush 2011-11-03 11:14:09.670033 7fb269f70700 client.5563 check_caps on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) wanted - used - is_delayed=0 2011-11-03 11:14:09.670065 7fb269f70700 client.5563 cap_delay_requeue on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.670093 7fb269f70700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.670109 7fb269f70700 client.5563 delaying cap release 2011-11-03 11:14:09.670139 7fb269f70700 client.5563 put_inode on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.670160 7fb269f70700 inode.put on 0x41bc45b0 10000000986.head now 1 2011-11-03 11:14:09.670178 7fb269f70700 client.5563 close exit(10) 2011-11-03 11:14:09.670208 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.670229 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.671201 7fb269f70700 client.5563 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:09.671235 7fb269f70700 client.5563 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:09.671263 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp 2011-11-03 11:14:09.671279 7fb269f70700 client.5563 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.671319 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.671359 7fb269f70700 client.5563 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins 2011-11-03 11:14:09.671374 7fb269f70700 client.5563 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.671408 7fb269f70700 client.5563 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.671430 7fb269f70700 client.5563 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred 2011-11-03 11:14:09.671448 7fb269f70700 client.5563 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.671481 7fb269f70700 client.5563 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.671503 7fb269f70700 client.5563 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging 2011-11-03 11:14:09.671517 7fb269f70700 client.5563 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:39.660420 seq 2 2011-11-03 11:14:09.671550 7fb269f70700 client.5563 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.671572 7fb269f70700 client.5563 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins 2011-11-03 11:14:09.671589 7fb269f70700 client.5563 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.671624 7fb269f70700 client.5563 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.671645 7fb269f70700 client.5563 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging 2011-11-03 11:14:09.671659 7fb269f70700 client.5563 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.671693 7fb269f70700 client.5563 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.671715 7fb269f70700 client.5563 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 2011-11-03 11:14:09.671733 7fb269f70700 client.5563 _lookup have dn job_201111031113_0001 mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.671766 7fb269f70700 client.5563 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.671802 7fb269f70700 client.5563 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml 2011-11-03 11:14:09.671817 7fb269f70700 client.5563 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:39.667190 seq 3 2011-11-03 11:14:09.671851 7fb269f70700 client.5563 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.671873 7fb269f70700 client.5563 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.671890 7fb269f70700 client.5563 fill_stat on 10000000986 snap/devhead mode 0100644 mtime 2011-11-03 11:14:04.597264 ctime 2011-11-03 11:14:04.597264 2011-11-03 11:14:09.671905 7fb269f70700 client.5563 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:09.671925 7fb269f70700 client.5563 open enter(/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0,0) = 2011-11-03 11:14:09.671943 7fb269f70700 client.5563 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:09.671964 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp 2011-11-03 11:14:09.671978 7fb269f70700 client.5563 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.672017 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.672039 7fb269f70700 client.5563 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins 2011-11-03 11:14:09.672053 7fb269f70700 client.5563 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.672087 7fb269f70700 client.5563 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.672108 7fb269f70700 client.5563 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred 2011-11-03 11:14:09.672122 7fb269f70700 client.5563 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.672159 7fb269f70700 client.5563 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.672180 7fb269f70700 client.5563 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging 2011-11-03 11:14:09.672195 7fb269f70700 client.5563 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:39.660420 seq 2 2011-11-03 11:14:09.672226 7fb269f70700 client.5563 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.672262 7fb269f70700 client.5563 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins 2011-11-03 11:14:09.672276 7fb269f70700 client.5563 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.672313 7fb269f70700 client.5563 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.672336 7fb269f70700 client.5563 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging 2011-11-03 11:14:09.672349 7fb269f70700 client.5563 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.672383 7fb269f70700 client.5563 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.672405 7fb269f70700 client.5563 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 2011-11-03 11:14:09.672419 7fb269f70700 client.5563 _lookup have dn job_201111031113_0001 mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.672456 7fb269f70700 client.5563 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.672477 7fb269f70700 client.5563 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml 2011-11-03 11:14:09.672492 7fb269f70700 client.5563 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:39.667190 seq 3 2011-11-03 11:14:09.672525 7fb269f70700 client.5563 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.672555 7fb269f70700 client.5563 check_caps on 10000000986.head(ref=1 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) wanted pFscr used - is_delayed=1 2011-11-03 11:14:09.672575 7fb269f70700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.672592 7fb269f70700 inode.get on 0x41bc45b0 10000000986.head now 2 2011-11-03 11:14:09.672604 7fb269f70700 client.5563 _create_fh 10000000986 mode 1 2011-11-03 11:14:09.672617 7fb269f70700 client.5563 trim_cache size 8 max 16384 2011-11-03 11:14:09.672634 7fb269f70700 client.5563 open exit(#1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0) = 11 2011-11-03 11:14:09.672654 7fb269f70700 client.5563 describe_layout(11) = 0 2011-11-03 11:14:09.672668 7fb269f70700 client.5563 close enter(11) 2011-11-03 11:14:09.672700 7fb269f70700 client.5563 _release_fh 0x7fb270631920 mode 1 on 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.672728 7fb269f70700 client.5563 _flush 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.672740 7fb269f70700 client.5563 nothing to flush 2011-11-03 11:14:09.672764 7fb269f70700 client.5563 check_caps on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) wanted - used - is_delayed=0 2011-11-03 11:14:09.672787 7fb269f70700 client.5563 cap_delay_requeue on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.672808 7fb269f70700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.672821 7fb269f70700 client.5563 delaying cap release 2011-11-03 11:14:09.672844 7fb269f70700 client.5563 put_inode on 10000000986.head(ref=2 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.672856 7fb269f70700 inode.put on 0x41bc45b0 10000000986.head now 1 2011-11-03 11:14:09.672871 7fb269f70700 client.5563 close exit(11) 2011-11-03 11:14:09.672916 7fb269f70700 client.5563 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:09.672933 7fb269f70700 client.5563 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:09.672956 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp 2011-11-03 11:14:09.672970 7fb269f70700 client.5563 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.673004 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.673030 7fb269f70700 client.5563 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins 2011-11-03 11:14:09.673044 7fb269f70700 client.5563 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.673078 7fb269f70700 client.5563 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.673100 7fb269f70700 client.5563 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred 2011-11-03 11:14:09.673114 7fb269f70700 client.5563 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.673147 7fb269f70700 client.5563 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.673228 7fb269f70700 client.5563 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging 2011-11-03 11:14:09.673243 7fb269f70700 client.5563 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:39.660420 seq 2 2011-11-03 11:14:09.673275 7fb269f70700 client.5563 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.673302 7fb269f70700 client.5563 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins 2011-11-03 11:14:09.673316 7fb269f70700 client.5563 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.673350 7fb269f70700 client.5563 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.673372 7fb269f70700 client.5563 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging 2011-11-03 11:14:09.673385 7fb269f70700 client.5563 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.673419 7fb269f70700 client.5563 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.673444 7fb269f70700 client.5563 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 2011-11-03 11:14:09.673458 7fb269f70700 client.5563 _lookup have dn job_201111031113_0001 mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.673492 7fb269f70700 client.5563 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.673513 7fb269f70700 client.5563 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml 2011-11-03 11:14:09.673527 7fb269f70700 client.5563 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:39.667190 seq 3 2011-11-03 11:14:09.673561 7fb269f70700 client.5563 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.673578 7fb269f70700 client.5563 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.673597 7fb269f70700 client.5563 fill_stat on 10000000986 snap/devhead mode 0100644 mtime 2011-11-03 11:14:04.597264 ctime 2011-11-03 11:14:04.597264 2011-11-03 11:14:09.673617 7fb269f70700 client.5563 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:09.673673 7fb269f70700 client.5563 open enter(/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0,0) = 2011-11-03 11:14:09.673691 7fb269f70700 client.5563 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:09.673713 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp 2011-11-03 11:14:09.673730 7fb269f70700 client.5563 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.673765 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.673787 7fb269f70700 client.5563 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins 2011-11-03 11:14:09.673801 7fb269f70700 client.5563 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.673835 7fb269f70700 client.5563 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.673857 7fb269f70700 client.5563 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred 2011-11-03 11:14:09.673874 7fb269f70700 client.5563 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.673908 7fb269f70700 client.5563 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.673929 7fb269f70700 client.5563 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging 2011-11-03 11:14:09.673944 7fb269f70700 client.5563 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:39.660420 seq 2 2011-11-03 11:14:09.673975 7fb269f70700 client.5563 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.673997 7fb269f70700 client.5563 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins 2011-11-03 11:14:09.674011 7fb269f70700 client.5563 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.674048 7fb269f70700 client.5563 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.674070 7fb269f70700 client.5563 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging 2011-11-03 11:14:09.674094 7fb269f70700 client.5563 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.674142 7fb269f70700 client.5563 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.674169 7fb269f70700 client.5563 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 2011-11-03 11:14:09.674183 7fb269f70700 client.5563 _lookup have dn job_201111031113_0001 mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.674217 7fb269f70700 client.5563 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.674238 7fb269f70700 client.5563 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml 2011-11-03 11:14:09.674253 7fb269f70700 client.5563 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:39.667190 seq 3 2011-11-03 11:14:09.674286 7fb269f70700 client.5563 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml = 10000000986.head(ref=1 cap_refs={} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.674318 7fb269f70700 client.5563 check_caps on 10000000986.head(ref=1 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) wanted pFscr used - is_delayed=1 2011-11-03 11:14:09.674337 7fb269f70700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.674350 7fb269f70700 inode.get on 0x41bc45b0 10000000986.head now 2 2011-11-03 11:14:09.674362 7fb269f70700 client.5563 _create_fh 10000000986 mode 1 2011-11-03 11:14:09.674374 7fb269f70700 client.5563 trim_cache size 8 max 16384 2011-11-03 11:14:09.674389 7fb269f70700 client.5563 open exit(#1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0) = 12 2011-11-03 11:14:09.674420 7fb269f70700 client.5563 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:09.674436 7fb269f70700 client.5563 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:09.674462 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp 2011-11-03 11:14:09.674478 7fb269f70700 client.5563 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.674512 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.674534 7fb269f70700 client.5563 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins 2011-11-03 11:14:09.674548 7fb269f70700 client.5563 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.674591 7fb269f70700 client.5563 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.674627 7fb269f70700 client.5563 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred 2011-11-03 11:14:09.674641 7fb269f70700 client.5563 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.674675 7fb269f70700 client.5563 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.674696 7fb269f70700 client.5563 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging 2011-11-03 11:14:09.674711 7fb269f70700 client.5563 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:39.660420 seq 2 2011-11-03 11:14:09.674746 7fb269f70700 client.5563 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.674769 7fb269f70700 client.5563 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins 2011-11-03 11:14:09.674783 7fb269f70700 client.5563 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.674817 7fb269f70700 client.5563 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.674838 7fb269f70700 client.5563 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging 2011-11-03 11:14:09.674852 7fb269f70700 client.5563 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.674889 7fb269f70700 client.5563 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.674912 7fb269f70700 client.5563 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 2011-11-03 11:14:09.674926 7fb269f70700 client.5563 _lookup have dn job_201111031113_0001 mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.674959 7fb269f70700 client.5563 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.674981 7fb269f70700 client.5563 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml 2011-11-03 11:14:09.675001 7fb269f70700 client.5563 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:39.667190 seq 3 2011-11-03 11:14:09.675046 7fb269f70700 client.5563 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml = 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.675064 7fb269f70700 client.5563 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.675080 7fb269f70700 client.5563 fill_stat on 10000000986 snap/devhead mode 0100644 mtime 2011-11-03 11:14:04.597264 ctime 2011-11-03 11:14:04.597264 2011-11-03 11:14:09.675094 7fb269f70700 client.5563 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:09.675112 7fb269f70700 client.5563 open enter(/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0,0) = 2011-11-03 11:14:09.675128 7fb269f70700 client.5563 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:09.675150 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp 2011-11-03 11:14:09.675167 7fb269f70700 client.5563 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.675201 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.675223 7fb269f70700 client.5563 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins 2011-11-03 11:14:09.675236 7fb269f70700 client.5563 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.675271 7fb269f70700 client.5563 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.675292 7fb269f70700 client.5563 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred 2011-11-03 11:14:09.675309 7fb269f70700 client.5563 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.675343 7fb269f70700 client.5563 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.675364 7fb269f70700 client.5563 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging 2011-11-03 11:14:09.675379 7fb269f70700 client.5563 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:39.660420 seq 2 2011-11-03 11:14:09.675410 7fb269f70700 client.5563 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.675432 7fb269f70700 client.5563 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins 2011-11-03 11:14:09.675462 7fb269f70700 client.5563 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.675496 7fb269f70700 client.5563 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.675518 7fb269f70700 client.5563 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging 2011-11-03 11:14:09.675532 7fb269f70700 client.5563 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.675566 7fb269f70700 client.5563 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.675591 7fb269f70700 client.5563 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 2011-11-03 11:14:09.675605 7fb269f70700 client.5563 _lookup have dn job_201111031113_0001 mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.675639 7fb269f70700 client.5563 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.675660 7fb269f70700 client.5563 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml 2011-11-03 11:14:09.675675 7fb269f70700 client.5563 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:39.667190 seq 3 2011-11-03 11:14:09.675709 7fb269f70700 client.5563 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml = 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.675740 7fb269f70700 client.5563 check_caps on 10000000986.head(ref=2 cap_refs={} open={1=2} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) wanted pFscr used - is_delayed=1 2011-11-03 11:14:09.675760 7fb269f70700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.675773 7fb269f70700 inode.get on 0x41bc45b0 10000000986.head now 3 2011-11-03 11:14:09.675785 7fb269f70700 client.5563 _create_fh 10000000986 mode 1 2011-11-03 11:14:09.675797 7fb269f70700 client.5563 trim_cache size 8 max 16384 2011-11-03 11:14:09.675812 7fb269f70700 client.5563 open exit(#1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml, 0) = 13 2011-11-03 11:14:09.675825 7fb269f70700 client.5563 describe_layout(13) = 0 2011-11-03 11:14:09.675838 7fb269f70700 client.5563 close enter(13) 2011-11-03 11:14:09.675861 7fb269f70700 client.5563 _release_fh 0x7fb270736ef0 mode 1 on 10000000986.head(ref=3 cap_refs={} open={1=2} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.675895 7fb269f70700 client.5563 put_inode on 10000000986.head(ref=3 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.675916 7fb269f70700 inode.put on 0x41bc45b0 10000000986.head now 2 2011-11-03 11:14:09.675928 7fb269f70700 client.5563 close exit(13) 2011-11-03 11:14:09.675962 7fb269f70700 client.5563 lstat enter (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:09.675979 7fb269f70700 client.5563 path_walk #1/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml 2011-11-03 11:14:09.676001 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp 2011-11-03 11:14:09.676015 7fb269f70700 client.5563 _lookup have dn tmp mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.676053 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) tmp = 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:09.676074 7fb269f70700 client.5563 1 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins 2011-11-03 11:14:09.676088 7fb269f70700 client.5563 _lookup have dn hadoop-nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.676122 7fb269f70700 client.5563 _lookup 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) hadoop-nwatkins = 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:09.676144 7fb269f70700 client.5563 2 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred 2011-11-03 11:14:09.676157 7fb269f70700 client.5563 _lookup have dn mapred mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.676195 7fb269f70700 client.5563 _lookup 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) mapred = 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:09.676216 7fb269f70700 client.5563 3 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging 2011-11-03 11:14:09.676231 7fb269f70700 client.5563 _lookup have dn staging mds.0 ttl 2011-11-03 11:14:39.660420 seq 2 2011-11-03 11:14:09.676262 7fb269f70700 client.5563 _lookup 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) staging = 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:09.676284 7fb269f70700 client.5563 4 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins 2011-11-03 11:14:09.676298 7fb269f70700 client.5563 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.676335 7fb269f70700 client.5563 _lookup 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) nwatkins = 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:09.676372 7fb269f70700 client.5563 5 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging 2011-11-03 11:14:09.676386 7fb269f70700 client.5563 _lookup have dn .staging mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.676420 7fb269f70700 client.5563 _lookup 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) .staging = 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:09.676442 7fb269f70700 client.5563 6 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 2011-11-03 11:14:09.676459 7fb269f70700 client.5563 _lookup have dn job_201111031113_0001 mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.676493 7fb269f70700 client.5563 _lookup 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) job_201111031113_0001 = 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:09.676514 7fb269f70700 client.5563 7 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml 2011-11-03 11:14:09.676528 7fb269f70700 client.5563 _lookup have dn job.xml mds.0 ttl 2011-11-03 11:14:39.667190 seq 3 2011-11-03 11:14:09.676562 7fb269f70700 client.5563 _lookup 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) job.xml = 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.676579 7fb269f70700 client.5563 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.676598 7fb269f70700 client.5563 fill_stat on 10000000986 snap/devhead mode 0100644 mtime 2011-11-03 11:14:04.597264 ctime 2011-11-03 11:14:04.597264 2011-11-03 11:14:09.676612 7fb269f70700 client.5563 lstat exit (relpath/tmp/hadoop-nwatkins/mapred/staging/nwatkins/.staging/job_201111031113_0001/job.xml mask 341) 2011-11-03 11:14:09.678705 7fb269f70700 client.5563 get_caps 10000000986.head(ref=2 cap_refs={} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:09.678729 7fb269f70700 client.5563 lock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.678758 7fb269f70700 client.5563 _read_async 10000000986.head(ref=2 cap_refs={2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 0~4096 2011-11-03 11:14:09.678775 7fb269f70700 client.5563 readahead=1 nr_consec=0 max_byes=0 max_periods=1 2011-11-03 11:14:09.678861 7fb269f70700 -- 192.168.141.130:0/3011511 connect_rank to 192.168.141.132:6800/3579, creating pipe and registering 2011-11-03 11:14:09.678990 7fb269f70700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=-1 pgs=0 cs=0 l=0).register_pipe 2011-11-03 11:14:09.679036 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=-1 pgs=0 cs=0 l=0).writer: state = 1 policy.server=0 2011-11-03 11:14:09.679072 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=-1 pgs=0 cs=0 l=0).connect 0 2011-11-03 11:14:09.679121 7fb269f70700 -- 192.168.141.130:0/3011511 --> 192.168.141.132:6800/3579 -- osd_op(client.5563.0:1 10000000986.00000000 [read 0~4096] 0.5d1b5ff1) v1 -- ?+0 0x7fb270988b90 con 0x7fb270736880 2011-11-03 11:14:09.679151 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message osd_op(client.5563.0:1 10000000986.00000000 [read 0~4096] 0.5d1b5ff1) v1 remote 192.168.141.132:6800/3579 2011-11-03 11:14:09.679180 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=0 cs=0 l=0).connecting to 192.168.141.132:6800/3579 2011-11-03 11:14:09.679685 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=0 cs=0 l=0).connect read peer addr 192.168.141.132:6800/3579 on socket 92 2011-11-03 11:14:09.679709 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=0 cs=0 l=0).connect peer addr for me is 192.168.141.130:50842/0 2011-11-03 11:14:09.679744 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=0 cs=0 l=0).connect sent my addr 192.168.141.130:0/3011511 2011-11-03 11:14:09.679833 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=0 cs=0 l=0).connect.authorizer_len=174 protocol=2 2011-11-03 11:14:09.679855 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=0 cs=0 l=0).connect sending gseq=3 cseq=0 proto=24 2011-11-03 11:14:09.679882 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=0 cs=0 l=0).connect wrote (self +) cseq, waiting for reply 2011-11-03 11:14:09.680205 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=0 cs=0 l=0).connect got reply tag 1 connect_seq 1 global_seq 579 proto 24 flags 1 2011-11-03 11:14:09.680224 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=0 cs=0 l=0).reply.authorizer_len=36 2011-11-03 11:14:09.680280 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).connect success 1, lossy = 1, features 4090 2011-11-03 11:14:09.680300 7fb262255700 -- 192.168.141.130:0/3011511 >> :/0 pipe(0x41ae1990 sd=-1 pgs=0 cs=0 l=0).queue_received queuing pipe 2011-11-03 11:14:09.680334 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).connect starting reader 2011-11-03 11:14:09.680360 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x41ae1990 dequeued 0 2011-11-03 11:14:09.680383 7fb263e5c700 client.5563 ms_handle_connect on 192.168.141.132:6800/3579 2011-11-03 11:14:09.680414 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.680447 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer encoding 1 0x7fb270988b90 osd_op(client.5563.0:1 10000000986.00000000 [read 0~4096] 0.5d1b5ff1) v1 2011-11-03 11:14:09.680483 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.680519 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer sending 1 0x7fb270988b90 2011-11-03 11:14:09.680543 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).write_message 0x7fb270988b90 2011-11-03 11:14:09.680574 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.680592 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.681077 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got ACK 2011-11-03 11:14:09.681115 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.681137 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got MSG 2011-11-03 11:14:09.681162 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got envelope type=43 src osd.7 front=111 data=4096 off 0 2011-11-03 11:14:09.681184 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader wants 4207 from dispatch throttler 0/104857600 2011-11-03 11:14:09.681209 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got front 111 2011-11-03 11:14:09.681231 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader allocating new rx buffer at offset 0 2011-11-03 11:14:09.681252 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader reading nonblocking into 0x41bc7000 len 4096 2011-11-03 11:14:09.681282 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).aborted = 0 2011-11-03 11:14:09.681299 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got 111 + 0 + 4096 byte message 2011-11-03 11:14:09.681343 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got message 1 0x41cc5a20 osd_op_reply(1 10000000986.00000000 [read 0~4096] = 0) v2 2011-11-03 11:14:09.681368 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:09.681394 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.681420 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb270736000 dequeued 0x41cc5a20 2011-11-03 11:14:09.681454 7fb263e5c700 -- 192.168.141.130:0/3011511 <== osd.7 192.168.141.132:6800/3579 1 ==== osd_op_reply(1 10000000986.00000000 [read 0~4096] = 0) v2 ==== 111+0+4096 (344334805 0 862425587) 0x41cc5a20 con 0x7fb270736880 2011-11-03 11:14:09.681482 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.681506 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).write_ack 1 2011-11-03 11:14:09.681532 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.681553 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.681579 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 4207 to dispatch throttler 4207/104857600 2011-11-03 11:14:09.681602 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41cc5a20 2011-11-03 11:14:09.681625 7fb269f70700 client.5563 unlock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.681643 7fb269f70700 client.5563 readahead nr_consec_read 1 for 4096 bytes .. last_pos 0 .. offset 0 2011-11-03 11:14:09.681658 7fb269f70700 client.5563 read(12, 0x7fb270987570, 4096, -1) = 4096 2011-11-03 11:14:09.681824 7fb269f70700 client.5563 get_caps 10000000986.head(ref=2 cap_refs={1024=0,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:09.681854 7fb269f70700 client.5563 lock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.681879 7fb269f70700 client.5563 _read_async 10000000986.head(ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 4096~4096 2011-11-03 11:14:09.681893 7fb269f70700 client.5563 readahead=1 nr_consec=1 max_byes=0 max_periods=1 2011-11-03 11:14:09.681910 7fb269f70700 client.5563 readahead 1 reads 4096 bytes ... readahead 4096~13014 min 4096 (caller wants 4096~4096) 2011-11-03 11:14:09.681963 7fb269f70700 -- 192.168.141.130:0/3011511 --> 192.168.141.132:6800/3579 -- osd_op(client.5563.0:2 10000000986.00000000 [read 4096~13014] 0.5d1b5ff1) v1 -- ?+0 0x7fb270988d00 con 0x7fb270736880 2011-11-03 11:14:09.681983 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message osd_op(client.5563.0:2 10000000986.00000000 [read 4096~13014] 0.5d1b5ff1) v1 remote 192.168.141.132:6800/3579 2011-11-03 11:14:09.682002 7fb269f70700 client.5563 readahead initiated 2011-11-03 11:14:09.682031 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.682062 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer encoding 2 0x7fb270988d00 osd_op(client.5563.0:2 10000000986.00000000 [read 4096~13014] 0.5d1b5ff1) v1 2011-11-03 11:14:09.682087 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer sending 2 0x7fb270988d00 2011-11-03 11:14:09.682105 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).write_message 0x7fb270988d00 2011-11-03 11:14:09.682134 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.682151 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.682505 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got ACK 2011-11-03 11:14:09.682532 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.682555 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got MSG 2011-11-03 11:14:09.682578 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got envelope type=43 src osd.7 front=111 data=13014 off 4096 2011-11-03 11:14:09.682595 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader wants 13125 from dispatch throttler 0/104857600 2011-11-03 11:14:09.682656 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got front 111 2011-11-03 11:14:09.682679 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader allocating new rx buffer at offset 0 2011-11-03 11:14:09.682703 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader reading nonblocking into 0x417a7000 len 12288 2011-11-03 11:14:09.682742 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader reading nonblocking into 0x417a60c0 len 726 2011-11-03 11:14:09.682775 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).aborted = 0 2011-11-03 11:14:09.682807 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got 111 + 0 + 13014 byte message 2011-11-03 11:14:09.682848 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader got message 2 0x417a6430 osd_op_reply(2 10000000986.00000000 [read 4096~13014] = 0) v2 2011-11-03 11:14:09.682871 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).queue_received queuing pipe 2011-11-03 11:14:09.682896 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader reading tag... 2011-11-03 11:14:09.682928 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb270736000 dequeued 0x417a6430 2011-11-03 11:14:09.682959 7fb263e5c700 -- 192.168.141.130:0/3011511 <== osd.7 192.168.141.132:6800/3579 2 ==== osd_op_reply(2 10000000986.00000000 [read 4096~13014] = 0) v2 ==== 111+0+13014 (3557038930 0 770417814) 0x417a6430 con 0x7fb270736880 2011-11-03 11:14:09.682988 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.683016 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).write_ack 2 2011-11-03 11:14:09.683041 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.683062 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.683084 7fb269f70700 client.5563 unlock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.683106 7fb269f70700 client.5563 readahead nr_consec_read 2 for 8192 bytes .. last_pos 4096 .. offset 4096 2011-11-03 11:14:09.683124 7fb269f70700 client.5563 read(12, 0x7fb270987570, 4096, -1) = 4096 2011-11-03 11:14:09.683149 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 13125 to dispatch throttler 13125/104857600 2011-11-03 11:14:09.683169 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x417a6430 2011-11-03 11:14:09.683278 7fb269f70700 client.5563 get_caps 10000000986.head(ref=2 cap_refs={1024=0,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:09.683297 7fb269f70700 client.5563 lock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.683321 7fb269f70700 client.5563 _read_async 10000000986.head(ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 8192~4096 2011-11-03 11:14:09.683337 7fb269f70700 client.5563 readahead=1 nr_consec=2 max_byes=0 max_periods=1 2011-11-03 11:14:09.683350 7fb269f70700 client.5563 readahead 2 reads 8192 bytes ... readahead 8192~8918 min 4096 (caller wants 8192~4096) 2011-11-03 11:14:09.683368 7fb269f70700 client.5563 readahead already have min 2011-11-03 11:14:09.683395 7fb269f70700 client.5563 unlock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.683408 7fb269f70700 client.5563 readahead nr_consec_read 3 for 12288 bytes .. last_pos 8192 .. offset 8192 2011-11-03 11:14:09.683421 7fb269f70700 client.5563 read(12, 0x7fb270987570, 4096, -1) = 4096 2011-11-03 11:14:09.683552 7fb269f70700 client.5563 get_caps 10000000986.head(ref=2 cap_refs={1024=0,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:09.683574 7fb269f70700 client.5563 lock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.683606 7fb269f70700 client.5563 _read_async 10000000986.head(ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 12288~4096 2011-11-03 11:14:09.683624 7fb269f70700 client.5563 readahead=1 nr_consec=3 max_byes=0 max_periods=1 2011-11-03 11:14:09.683637 7fb269f70700 client.5563 readahead 3 reads 12288 bytes ... readahead 12288~4822 min 2411 (caller wants 12288~4096) 2011-11-03 11:14:09.683654 7fb269f70700 client.5563 readahead already have min 2011-11-03 11:14:09.683674 7fb269f70700 client.5563 unlock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.683688 7fb269f70700 client.5563 readahead nr_consec_read 4 for 16384 bytes .. last_pos 12288 .. offset 12288 2011-11-03 11:14:09.683700 7fb269f70700 client.5563 read(12, 0x7fb270987570, 4096, -1) = 4096 2011-11-03 11:14:09.683830 7fb269f70700 client.5563 get_caps 10000000986.head(ref=2 cap_refs={1024=0,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:09.683845 7fb269f70700 client.5563 lock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.683869 7fb269f70700 client.5563 _read_async 10000000986.head(ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 16384~4096 2011-11-03 11:14:09.683882 7fb269f70700 client.5563 readahead=0 nr_consec=4 max_byes=0 max_periods=1 2011-11-03 11:14:09.683906 7fb269f70700 client.5563 unlock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.683919 7fb269f70700 client.5563 readahead nr_consec_read 5 for 17110 bytes .. last_pos 16384 .. offset 16384 2011-11-03 11:14:09.683932 7fb269f70700 client.5563 read(12, 0x7fb270987570, 4096, -1) = 726 2011-11-03 11:14:09.683971 7fb269f70700 client.5563 get_caps 10000000986.head(ref=2 cap_refs={1024=0,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) have pAsLsXsFscr need Fr want Fc but not Fc revoking - 2011-11-03 11:14:09.683986 7fb269f70700 client.5563 lock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.684010 7fb269f70700 client.5563 _read_async 10000000986.head(ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 17110~4096 2011-11-03 11:14:09.684022 7fb269f70700 client.5563 unlock_fh_pos 0x7fb270631920 2011-11-03 11:14:09.684034 7fb269f70700 client.5563 readahead nr_consec_read 6 for 17110 bytes .. last_pos 17110 .. offset 17110 2011-11-03 11:14:09.684050 7fb269f70700 client.5563 read(12, 0x7fb270987570, 4096, -1) = 0 2011-11-03 11:14:09.684269 7fb269f70700 client.5563 _fsync(0x7fb270631920, data+metadata) 2011-11-03 11:14:09.684299 7fb269f70700 client.5563 _flush 10000000986.head(ref=2 cap_refs={1024=-1,2048=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.684312 7fb269f70700 client.5563 nothing to flush 2011-11-03 11:14:09.684324 7fb269f70700 client.5563 no metadata needs to commit 2011-11-03 11:14:09.684342 7fb269f70700 client.5563 wait_sync_caps want 0 (last is 0, 0 total flushing) 2011-11-03 11:14:09.684355 7fb269f70700 client.5563 ino 10000000986 has no uncommitted writes 2011-11-03 11:14:09.684367 7fb269f70700 client.5563 fsync(12, 0) = 0 2011-11-03 11:14:09.684387 7fb269f70700 client.5563 close enter(12) 2011-11-03 11:14:09.684419 7fb269f70700 client.5563 _release_fh 0x7fb270631920 mode 1 on 10000000986.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=1} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.684452 7fb269f70700 client.5563 _flush 10000000986.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.684464 7fb269f70700 client.5563 nothing to flush 2011-11-03 11:14:09.684495 7fb269f70700 client.5563 check_caps on 10000000986.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) wanted Fc used Fc is_delayed=0 2011-11-03 11:14:09.684520 7fb269f70700 client.5563 cap_delay_requeue on 10000000986.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.684542 7fb269f70700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.684555 7fb269f70700 client.5563 delaying cap release 2011-11-03 11:14:09.684579 7fb269f70700 client.5563 put_inode on 10000000986.head(ref=2 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) 2011-11-03 11:14:09.684592 7fb269f70700 inode.put on 0x41bc45b0 10000000986.head now 1 2011-11-03 11:14:09.684604 7fb269f70700 client.5563 close exit(12) 2011-11-03 11:14:09.718854 7fb267468700 -- 192.168.141.130:0/2011511 send_keepalive con 0x41c05490, have pipe. 2011-11-03 11:14:09.718956 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.719024 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_keepalive 2011-11-03 11:14:09.719077 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:09.719094 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:09.728908 7fb269f70700 client.5563 lstat enter (relpath/user/nwatkins/in-dir/_partition.lst mask 341) 2011-11-03 11:14:09.728979 7fb269f70700 client.5563 path_walk #1/user/nwatkins/in-dir/_partition.lst 2011-11-03 11:14:09.729016 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) user 2011-11-03 11:14:09.729044 7fb269f70700 client.5563 _lookup on #1/user 2011-11-03 11:14:09.729076 7fb269f70700 client.5563 choose_target_mds starting with req->inode 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) 2011-11-03 11:14:09.729092 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on user => 2176124347 2011-11-03 11:14:09.729116 7fb269f70700 client.5563 choose_target_mds 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) is_hash=1 hash=2176124347 2011-11-03 11:14:09.729139 7fb269f70700 client.5563 choose_target_mds from caps on inode 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) 2011-11-03 11:14:09.729151 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.729166 7fb269f70700 client.5563 send_request rebuilding request 10 for mds.0 2011-11-03 11:14:09.729200 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb270737070, mds: 0) 2011-11-03 11:14:09.729234 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.729229 2011-11-03 11:14:09.729253 7fb269f70700 client.5563 send_request client_request(unknown.0:10 lookup #1/user) v1 to mds.0 2011-11-03 11:14:09.729277 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:10 lookup #1/user) v1 -- ?+0 0x7fb2707373d0 2011-11-03 11:14:09.729301 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:10 lookup #1/user) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.729331 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6e730 2011-11-03 11:14:09.729374 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.729447 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 13 0x7fb2707373d0 client_request(client.5563:10 lookup #1/user) v1 2011-11-03 11:14:09.729481 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 13 0x7fb2707373d0 2011-11-03 11:14:09.729547 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb2707373d0 2011-11-03 11:14:09.729587 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.729604 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.729817 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.729856 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.729878 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.729903 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.729929 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.729956 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=583 data=0 off 0 2011-11-03 11:14:09.729974 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 583 from dispatch throttler 0/104857600 2011-11-03 11:14:09.730001 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 583 2011-11-03 11:14:09.730025 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.730042 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 583 + 0 + 0 byte message 2011-11-03 11:14:09.730093 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 12 0x41bc6ca0 client_reply(???:10 = 0 Success) v1 2011-11-03 11:14:09.730118 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.730146 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.730189 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41bc6ca0 2011-11-03 11:14:09.730223 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 12 ==== client_reply(???:10 = 0 Success) v1 ==== 583+0+0 (3718902978 0 0) 0x41bc6ca0 con 0x7fb27061cef0 2011-11-03 11:14:09.730243 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 10 2011-11-03 11:14:09.730262 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.729229 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.730279 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.730296 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.730316 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=11 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.730334 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.730366 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.730395 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 12 2011-11-03 11:14:09.730423 7fb263e5c700 client.5563 add_update_inode adding 10000000ce0.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x417a6430) caps pAsLsXsFs 2011-11-03 11:14:09.730446 7fb263e5c700 client.5563 dir hash is 2 2011-11-03 11:14:09.730467 7fb263e5c700 client.5563 update_inode_file_bits 10000000ce0.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x417a6430) - mtime 2011-11-01 21:12:56.017215 2011-11-03 11:14:09.730486 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.730521 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXsFs from mds.0 on 10000000ce0.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) 0x417a6430) 2011-11-03 11:14:09.730550 7fb263e5c700 client.5563 add_update_inode had 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) caps pAsLsXsFs 2011-11-03 11:14:09.730584 7fb263e5c700 client.5563 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) 2011-11-03 11:14:09.730603 7fb263e5c700 client.5563 got dirfrag map for 1 frag * to mds 0 2011-11-03 11:14:09.730622 7fb263e5c700 client.5563 insert_dentry_inode 'user' vino 10000000ce0.head in dir 1.head dn 0 2011-11-03 11:14:09.730639 7fb263e5c700 inode.get on 0x417a6430 10000000ce0.head now 1 2011-11-03 11:14:09.730658 7fb263e5c700 client.5563 link dir 0x41ae6510 'user' to inode 0x417a6430 dn 0x41cc47d0 (new dn) 2011-11-03 11:14:09.730677 7fb263e5c700 inode.get on 0x417a6430 10000000ce0.head now 2 2011-11-03 11:14:09.730695 7fb263e5c700 client.5563 link inode 0x417a6430 parents now 0x41cc47d0 2011-11-03 11:14:09.730712 7fb263e5c700 inode.put on 0x417a6430 10000000ce0.head now 1 2011-11-03 11:14:09.730729 7fb263e5c700 client.5563 setting dn offset to 3 2011-11-03 11:14:09.730749 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6e730 2011-11-03 11:14:09.730769 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 10 0x7fb263e5b490 2011-11-03 11:14:09.730799 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.730824 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.730848 7fb269f70700 client.5563 sendrecv kickback on tid 10 0x7fb263e5b490 2011-11-03 11:14:09.730875 7fb269f70700 client.5563 lat 0.001643 2011-11-03 11:14:09.730892 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.730937 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) user = 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) 2011-11-03 11:14:09.730960 7fb269f70700 client.5563 1 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) nwatkins 2011-11-03 11:14:09.730984 7fb269f70700 client.5563 _lookup on #10000000ce0/nwatkins 2011-11-03 11:14:09.731008 7fb269f70700 client.5563 choose_target_mds starting with req->inode 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) 2011-11-03 11:14:09.731022 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on nwatkins => 2332913853 2011-11-03 11:14:09.731045 7fb269f70700 client.5563 choose_target_mds 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) is_hash=1 hash=2332913853 2011-11-03 11:14:09.731069 7fb269f70700 client.5563 choose_target_mds from caps on inode 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) 2011-11-03 11:14:09.731081 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.731093 7fb269f70700 client.5563 send_request rebuilding request 11 for mds.0 2011-11-03 11:14:09.731109 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb270737340, mds: 0) 2011-11-03 11:14:09.731125 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.731121 2011-11-03 11:14:09.731141 7fb269f70700 client.5563 send_request client_request(unknown.0:11 lookup #10000000ce0/nwatkins) v1 to mds.0 2011-11-03 11:14:09.731160 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:11 lookup #10000000ce0/nwatkins) v1 -- ?+0 0x7fb270987570 2011-11-03 11:14:09.731179 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:11 lookup #10000000ce0/nwatkins) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.731199 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6e730 2011-11-03 11:14:09.731226 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 583 to dispatch throttler 583/104857600 2011-11-03 11:14:09.731245 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41bc6ca0 2011-11-03 11:14:09.731271 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.731300 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 14 0x7fb270987570 client_request(client.5563:11 lookup #10000000ce0/nwatkins) v1 2011-11-03 11:14:09.731326 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 14 0x7fb270987570 2011-11-03 11:14:09.731346 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270987570 2011-11-03 11:14:09.731384 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.731401 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.731666 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.731699 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.731730 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.731753 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=587 data=0 off 0 2011-11-03 11:14:09.731769 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 587 from dispatch throttler 0/104857600 2011-11-03 11:14:09.731796 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 587 2011-11-03 11:14:09.731819 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.731835 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 587 + 0 + 0 byte message 2011-11-03 11:14:09.731861 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 13 0x41bc6ca0 client_reply(???:11 = 0 Success) v1 2011-11-03 11:14:09.731880 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.731902 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.731929 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41bc6ca0 2011-11-03 11:14:09.731958 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 13 ==== client_reply(???:11 = 0 Success) v1 ==== 587+0+0 (1933354525 0 0) 0x41bc6ca0 con 0x7fb27061cef0 2011-11-03 11:14:09.731971 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 11 2011-11-03 11:14:09.731985 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.731121 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.732010 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.732022 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.732037 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=12 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.732050 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.732074 7fb263e5c700 client.5563 add_update_inode adding 10000000ce1.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x417a6870) caps pAsLsXsFs 2011-11-03 11:14:09.732087 7fb263e5c700 client.5563 dir hash is 2 2011-11-03 11:14:09.732103 7fb263e5c700 client.5563 update_inode_file_bits 10000000ce1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x417a6870) - mtime 2011-11-03 11:14:05.347177 2011-11-03 11:14:09.732116 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.732144 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXsFs from mds.0 on 10000000ce1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) 0x417a6870) 2011-11-03 11:14:09.732168 7fb263e5c700 client.5563 add_update_inode had 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) caps pAsLsXsFs 2011-11-03 11:14:09.732196 7fb263e5c700 client.5563 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) 2011-11-03 11:14:09.732216 7fb263e5c700 client.5563 got dirfrag map for 10000000ce0 frag * to mds -1 2011-11-03 11:14:09.732232 7fb263e5c700 client.5563 insert_dentry_inode 'nwatkins' vino 10000000ce1.head in dir 10000000ce0.head dn 0 2011-11-03 11:14:09.732253 7fb263e5c700 inode.get on 0x417a6870 10000000ce1.head now 1 2011-11-03 11:14:09.732267 7fb263e5c700 client.5563 link dir 0x417a6430 'nwatkins' to inode 0x417a6870 dn 0x41bc6f80 (new dn) 2011-11-03 11:14:09.732279 7fb263e5c700 inode.get on 0x417a6870 10000000ce1.head now 2 2011-11-03 11:14:09.732292 7fb263e5c700 client.5563 link inode 0x417a6870 parents now 0x41bc6f80 2011-11-03 11:14:09.732304 7fb263e5c700 inode.put on 0x417a6870 10000000ce1.head now 1 2011-11-03 11:14:09.732316 7fb263e5c700 client.5563 setting dn offset to 2 2011-11-03 11:14:09.732328 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6e730 2011-11-03 11:14:09.732343 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 11 0x7fb263e5b490 2011-11-03 11:14:09.732366 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.732384 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 13 2011-11-03 11:14:09.732408 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.732424 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.732443 7fb269f70700 client.5563 sendrecv kickback on tid 11 0x7fb263e5b490 2011-11-03 11:14:09.732458 7fb269f70700 client.5563 lat 0.001334 2011-11-03 11:14:09.732472 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.732506 7fb269f70700 client.5563 _lookup 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) nwatkins = 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) 2011-11-03 11:14:09.732528 7fb269f70700 client.5563 2 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) in-dir 2011-11-03 11:14:09.732546 7fb269f70700 client.5563 _lookup on #10000000ce1/in-dir 2011-11-03 11:14:09.732569 7fb269f70700 client.5563 choose_target_mds starting with req->inode 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) 2011-11-03 11:14:09.732582 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on in-dir => 3658178500 2011-11-03 11:14:09.732604 7fb269f70700 client.5563 choose_target_mds 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) is_hash=1 hash=3658178500 2011-11-03 11:14:09.732626 7fb269f70700 client.5563 choose_target_mds from caps on inode 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) 2011-11-03 11:14:09.732638 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.732650 7fb269f70700 client.5563 send_request rebuilding request 12 for mds.0 2011-11-03 11:14:09.732664 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb270737070, mds: 0) 2011-11-03 11:14:09.732679 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.732675 2011-11-03 11:14:09.732693 7fb269f70700 client.5563 send_request client_request(unknown.0:12 lookup #10000000ce1/in-dir) v1 to mds.0 2011-11-03 11:14:09.732712 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:12 lookup #10000000ce1/in-dir) v1 -- ?+0 0x7fb270987570 2011-11-03 11:14:09.732735 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:12 lookup #10000000ce1/in-dir) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.732768 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.732790 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 15 0x7fb270987570 client_request(client.5563:12 lookup #10000000ce1/in-dir) v1 2011-11-03 11:14:09.732815 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 15 0x7fb270987570 2011-11-03 11:14:09.732833 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270987570 2011-11-03 11:14:09.732864 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.732880 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.732895 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6e730 2011-11-03 11:14:09.732916 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 587 to dispatch throttler 587/104857600 2011-11-03 11:14:09.732930 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41bc6ca0 2011-11-03 11:14:09.733101 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.733133 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.733155 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.733178 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=585 data=0 off 0 2011-11-03 11:14:09.733195 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 585 from dispatch throttler 0/104857600 2011-11-03 11:14:09.733222 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 585 2011-11-03 11:14:09.733245 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.733261 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 585 + 0 + 0 byte message 2011-11-03 11:14:09.733289 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 14 0x41bc6ca0 client_reply(???:12 = 0 Success) v1 2011-11-03 11:14:09.733313 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.733337 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.733366 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.733389 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 14 2011-11-03 11:14:09.733409 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.733431 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.733458 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41bc6ca0 2011-11-03 11:14:09.733479 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 14 ==== client_reply(???:12 = 0 Success) v1 ==== 585+0+0 (1275374500 0 0) 0x41bc6ca0 con 0x7fb27061cef0 2011-11-03 11:14:09.733491 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 12 2011-11-03 11:14:09.733506 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.732675 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.733517 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.733529 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.733544 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=13 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.733556 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.733579 7fb263e5c700 client.5563 add_update_inode adding 10000000ce2.head(ref=0 cap_refs={} open={} mode=40000 size=0 mtime=0.000000 caps=- 0x41bc5610) caps pAsLsXsFs 2011-11-03 11:14:09.733592 7fb263e5c700 client.5563 dir hash is 2 2011-11-03 11:14:09.733608 7fb263e5c700 client.5563 update_inode_file_bits 10000000ce2.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=0.000000 caps=- 0x41bc5610) - mtime 2011-11-03 11:14:01.196477 2011-11-03 11:14:09.733621 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.733649 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXsFs from mds.0 on 10000000ce2.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41bc5610) 2011-11-03 11:14:09.733673 7fb263e5c700 client.5563 add_update_inode had 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) caps pAsLsXsFs 2011-11-03 11:14:09.733700 7fb263e5c700 client.5563 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) 2011-11-03 11:14:09.733713 7fb263e5c700 client.5563 got dirfrag map for 10000000ce1 frag * to mds -1 2011-11-03 11:14:09.733729 7fb263e5c700 client.5563 insert_dentry_inode 'in-dir' vino 10000000ce2.head in dir 10000000ce1.head dn 0 2011-11-03 11:14:09.733741 7fb263e5c700 inode.get on 0x41bc5610 10000000ce2.head now 1 2011-11-03 11:14:09.733755 7fb263e5c700 client.5563 link dir 0x417a6870 'in-dir' to inode 0x41bc5610 dn 0x417a6e70 (new dn) 2011-11-03 11:14:09.733768 7fb263e5c700 inode.get on 0x41bc5610 10000000ce2.head now 2 2011-11-03 11:14:09.733780 7fb263e5c700 client.5563 link inode 0x41bc5610 parents now 0x417a6e70 2011-11-03 11:14:09.733792 7fb263e5c700 inode.put on 0x41bc5610 10000000ce2.head now 1 2011-11-03 11:14:09.733804 7fb263e5c700 client.5563 setting dn offset to 2 2011-11-03 11:14:09.733817 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6e730 2011-11-03 11:14:09.733831 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 12 0x7fb263e5b490 2011-11-03 11:14:09.733850 7fb269f70700 client.5563 sendrecv kickback on tid 12 0x7fb263e5b490 2011-11-03 11:14:09.733865 7fb269f70700 client.5563 lat 0.001187 2011-11-03 11:14:09.733878 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.733913 7fb269f70700 client.5563 _lookup 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) in-dir = 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) 2011-11-03 11:14:09.733936 7fb269f70700 client.5563 3 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) _partition.lst 2011-11-03 11:14:09.733967 7fb269f70700 client.5563 _lookup on #10000000ce2/_partition.lst 2011-11-03 11:14:09.733992 7fb269f70700 client.5563 choose_target_mds starting with req->inode 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) 2011-11-03 11:14:09.734005 7fb269f70700 client.5563 choose_target_mds inode dir hash is 2 on _partition.lst => 1027117801 2011-11-03 11:14:09.734027 7fb269f70700 client.5563 choose_target_mds 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) is_hash=1 hash=1027117801 2011-11-03 11:14:09.734049 7fb269f70700 client.5563 choose_target_mds from caps on inode 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) 2011-11-03 11:14:09.734061 7fb269f70700 client.5563 mds is 0 2011-11-03 11:14:09.734073 7fb269f70700 client.5563 send_request rebuilding request 13 for mds.0 2011-11-03 11:14:09.734087 7fb269f70700 client.5563 encode_cap_releases enter (req: 0x7fb270737340, mds: 0) 2011-11-03 11:14:09.734103 7fb269f70700 client.5563 send_request set sent_stamp to 2011-11-03 11:14:09.734098 2011-11-03 11:14:09.734117 7fb269f70700 client.5563 send_request client_request(unknown.0:13 lookup #10000000ce2/_partition.lst) v1 to mds.0 2011-11-03 11:14:09.734135 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_request(client.5563:13 lookup #10000000ce2/_partition.lst) v1 -- ?+0 0x7fb270987570 2011-11-03 11:14:09.734153 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_request(client.5563:13 lookup #10000000ce2/_partition.lst) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.734179 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.734199 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 16 0x7fb270987570 client_request(client.5563:13 lookup #10000000ce2/_partition.lst) v1 2011-11-03 11:14:09.734220 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 16 0x7fb270987570 2011-11-03 11:14:09.734238 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270987570 2011-11-03 11:14:09.734265 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.734281 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.734296 7fb269f70700 client.5563 awaiting reply|forward|kick on 0x7fb269f6e730 2011-11-03 11:14:09.734317 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 585 to dispatch throttler 585/104857600 2011-11-03 11:14:09.734331 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41bc6ca0 2011-11-03 11:14:09.734532 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.734564 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.734586 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:09.734609 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=26 src mds.0 front=593 data=0 off 0 2011-11-03 11:14:09.734641 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 593 from dispatch throttler 0/104857600 2011-11-03 11:14:09.734668 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 593 2011-11-03 11:14:09.734691 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:09.734707 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 593 + 0 + 0 byte message 2011-11-03 11:14:09.734734 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 15 0x41bc5b30 client_reply(???:13 = 0 Success) v1 2011-11-03 11:14:09.734758 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:09.734782 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:09.734813 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.734837 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 15 2011-11-03 11:14:09.734858 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.734874 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.734892 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41bc5b30 2011-11-03 11:14:09.734913 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 15 ==== client_reply(???:13 = 0 Success) v1 ==== 593+0+0 (2625552806 0 0) 0x41bc5b30 con 0x7fb27061cef0 2011-11-03 11:14:09.734926 7fb263e5c700 client.5563 handle_client_reply got a reply. Safe:1 tid 13 2011-11-03 11:14:09.734941 7fb263e5c700 client.5563 insert_trace from 2011-11-03 11:14:09.734098 mds.0 is_target=1 is_dentry=1 2011-11-03 11:14:09.734953 7fb263e5c700 client.5563 features 0xffa 2011-11-03 11:14:09.734964 7fb263e5c700 client.5563 update_snap_trace len 48 2011-11-03 11:14:09.734979 7fb263e5c700 client.5563 update_snap_trace snaprealm(1 nref=14 c=0 seq=1 parent=0 my_snaps=[] cached_snapc=1=[]) seq 1 <= 1 and same parent, SKIPPING 2011-11-03 11:14:09.734992 7fb263e5c700 client.5563 hrm is_target=1 is_dentry=1 2011-11-03 11:14:09.735017 7fb263e5c700 client.5563 add_update_inode adding 10000000981.head(ref=0 cap_refs={} open={} mode=100000 size=0 mtime=0.000000 caps=- objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x41bc8660) caps pAsLsXsFscr 2011-11-03 11:14:09.735035 7fb263e5c700 client.5563 update_inode_file_bits 10000000981.head(ref=0 cap_refs={} open={} mode=100777 size=0 mtime=0.000000 caps=- objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x41bc8660) - mtime 2011-11-03 11:14:01.196477 2011-11-03 11:14:09.735048 7fb263e5c700 client.5563 size 0 -> 129 2011-11-03 11:14:09.735060 7fb263e5c700 client.5563 add_update_cap first one, opened snaprealm 0x41ae6430 2011-11-03 11:14:09.735089 7fb263e5c700 client.5563 add_update_cap issued - -> pAsLsXsFscr from mds.0 on 10000000981.head(ref=0 cap_refs={} open={} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x41bc8660) 2011-11-03 11:14:09.735113 7fb263e5c700 client.5563 add_update_inode had 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) caps pAsLsXsFs 2011-11-03 11:14:09.735155 7fb263e5c700 client.5563 add_update_cap issued pAsLsXsFs -> pAsLsXsFs from mds.0 on 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) 2011-11-03 11:14:09.735168 7fb263e5c700 client.5563 got dirfrag map for 10000000ce2 frag * to mds -1 2011-11-03 11:14:09.735184 7fb263e5c700 client.5563 insert_dentry_inode '_partition.lst' vino 10000000981.head in dir 10000000ce2.head dn 0 2011-11-03 11:14:09.735197 7fb263e5c700 inode.get on 0x41bc8660 10000000981.head now 1 2011-11-03 11:14:09.735211 7fb263e5c700 client.5563 link dir 0x41bc5610 '_partition.lst' to inode 0x41bc8660 dn 0x41bc5dd0 (new dn) 2011-11-03 11:14:09.735224 7fb263e5c700 inode.get on 0x41bc8660 10000000981.head now 2 2011-11-03 11:14:09.735237 7fb263e5c700 client.5563 link inode 0x41bc8660 parents now 0x41bc5dd0 2011-11-03 11:14:09.735249 7fb263e5c700 inode.put on 0x41bc8660 10000000981.head now 1 2011-11-03 11:14:09.735260 7fb263e5c700 client.5563 setting dn offset to 2 2011-11-03 11:14:09.735273 7fb263e5c700 client.5563 handle_client_reply signalling caller 0x7fb269f6e730 2011-11-03 11:14:09.735288 7fb263e5c700 client.5563 handle_client_reply awaiting kickback on tid 13 0x7fb263e5b490 2011-11-03 11:14:09.735307 7fb269f70700 client.5563 sendrecv kickback on tid 13 0x7fb263e5b490 2011-11-03 11:14:09.735321 7fb269f70700 client.5563 lat 0.001220 2011-11-03 11:14:09.735335 7fb269f70700 client.5563 _lookup res is 0 2011-11-03 11:14:09.735370 7fb269f70700 client.5563 _lookup 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) _partition.lst = 10000000981.head(ref=1 cap_refs={} open={} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.735390 7fb269f70700 client.5563 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.735406 7fb269f70700 client.5563 fill_stat on 10000000981 snap/devhead mode 0100777 mtime 2011-11-03 11:14:01.196477 ctime 2011-11-03 11:14:01.196477 2011-11-03 11:14:09.735421 7fb269f70700 client.5563 lstat exit (relpath/user/nwatkins/in-dir/_partition.lst mask 341) 2011-11-03 11:14:09.735442 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 593 to dispatch throttler 593/104857600 2011-11-03 11:14:09.735456 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41bc5b30 2011-11-03 11:14:09.735515 7fb269f70700 client.5563 open enter(/user/nwatkins/in-dir/_partition.lst, 0,0) = 2011-11-03 11:14:09.735535 7fb269f70700 client.5563 path_walk #1/user/nwatkins/in-dir/_partition.lst 2011-11-03 11:14:09.735557 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) user 2011-11-03 11:14:09.735571 7fb269f70700 client.5563 _lookup have dn user mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.735605 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) user = 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) 2011-11-03 11:14:09.735627 7fb269f70700 client.5563 1 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) nwatkins 2011-11-03 11:14:09.735641 7fb269f70700 client.5563 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.735676 7fb269f70700 client.5563 _lookup 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) nwatkins = 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) 2011-11-03 11:14:09.735711 7fb269f70700 client.5563 2 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) in-dir 2011-11-03 11:14:09.735724 7fb269f70700 client.5563 _lookup have dn in-dir mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.735758 7fb269f70700 client.5563 _lookup 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) in-dir = 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) 2011-11-03 11:14:09.735780 7fb269f70700 client.5563 3 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) _partition.lst 2011-11-03 11:14:09.735794 7fb269f70700 client.5563 _lookup have dn _partition.lst mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.735830 7fb269f70700 client.5563 _lookup 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) _partition.lst = 10000000981.head(ref=1 cap_refs={} open={} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.735861 7fb269f70700 client.5563 check_caps on 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) wanted pFscr used - is_delayed=1 2011-11-03 11:14:09.735881 7fb269f70700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.735914 7fb269f70700 client.5563 send_cap 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) mds.0 seq 1 used - want pFscr flush - retain pAsxLsxXsxFsxcrwbl held pAsLsXsFscr revoking - dropping - 2011-11-03 11:14:09.735928 7fb269f70700 client.5563 auth cap, setting max_size = 0 2011-11-03 11:14:09.735955 7fb269f70700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 10000000981 6778 seq 1 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 -- ?+0 0x7fb270737070 2011-11-03 11:14:09.735979 7fb269f70700 -- 192.168.141.130:0/3011511 submit_message client_caps(update ino 10000000981 6778 seq 1 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:09.736005 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.736031 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 17 0x7fb270737070 client_caps(update ino 10000000981 6778 seq 1 caps=pAsLsXsFscr dirty=- wanted=pFscr follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 2011-11-03 11:14:09.736064 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 17 0x7fb270737070 2011-11-03 11:14:09.736082 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x7fb270737070 2011-11-03 11:14:09.736109 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:09.736131 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:09.736156 7fb269f70700 inode.get on 0x41bc8660 10000000981.head now 2 2011-11-03 11:14:09.736168 7fb269f70700 client.5563 _create_fh 10000000981 mode 1 2011-11-03 11:14:09.736181 7fb269f70700 client.5563 trim_cache size 12 max 16384 2011-11-03 11:14:09.736197 7fb269f70700 client.5563 open exit(#1/user/nwatkins/in-dir/_partition.lst, 0) = 14 2011-11-03 11:14:09.736211 7fb269f70700 client.5563 describe_layout(14) = 0 2011-11-03 11:14:09.736224 7fb269f70700 client.5563 close enter(14) 2011-11-03 11:14:09.736249 7fb269f70700 client.5563 _release_fh 0x7fb270736d60 mode 1 on 10000000981.head(ref=2 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.736273 7fb269f70700 client.5563 _flush 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.736285 7fb269f70700 client.5563 nothing to flush 2011-11-03 11:14:09.736309 7fb269f70700 client.5563 check_caps on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) wanted - used - is_delayed=0 2011-11-03 11:14:09.736333 7fb269f70700 client.5563 cap_delay_requeue on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.736354 7fb269f70700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.736365 7fb269f70700 client.5563 delaying cap release 2011-11-03 11:14:09.736391 7fb269f70700 client.5563 put_inode on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.736404 7fb269f70700 inode.put on 0x41bc8660 10000000981.head now 1 2011-11-03 11:14:09.736416 7fb269f70700 client.5563 close exit(14) 2011-11-03 11:14:09.737260 7fb269f70700 client.5563 lstat enter (relpath/user/nwatkins/in-dir/_partition.lst mask 341) 2011-11-03 11:14:09.737286 7fb269f70700 client.5563 path_walk #1/user/nwatkins/in-dir/_partition.lst 2011-11-03 11:14:09.737312 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) user 2011-11-03 11:14:09.737327 7fb269f70700 client.5563 _lookup have dn user mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.737363 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) user = 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) 2011-11-03 11:14:09.737386 7fb269f70700 client.5563 1 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) nwatkins 2011-11-03 11:14:09.737399 7fb269f70700 client.5563 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.737434 7fb269f70700 client.5563 _lookup 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) nwatkins = 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) 2011-11-03 11:14:09.737466 7fb269f70700 client.5563 2 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) in-dir 2011-11-03 11:14:09.737489 7fb269f70700 client.5563 _lookup have dn in-dir mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.737524 7fb269f70700 client.5563 _lookup 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) in-dir = 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) 2011-11-03 11:14:09.737546 7fb269f70700 client.5563 3 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) _partition.lst 2011-11-03 11:14:09.737560 7fb269f70700 client.5563 _lookup have dn _partition.lst mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.737596 7fb269f70700 client.5563 _lookup 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) _partition.lst = 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.737614 7fb269f70700 client.5563 _getattr mask pAsLsXsFs issued=1 2011-11-03 11:14:09.737630 7fb269f70700 client.5563 fill_stat on 10000000981 snap/devhead mode 0100777 mtime 2011-11-03 11:14:01.196477 ctime 2011-11-03 11:14:01.196477 2011-11-03 11:14:09.737644 7fb269f70700 client.5563 lstat exit (relpath/user/nwatkins/in-dir/_partition.lst mask 341) 2011-11-03 11:14:09.737665 7fb269f70700 client.5563 open enter(/user/nwatkins/in-dir/_partition.lst, 0,0) = 2011-11-03 11:14:09.737680 7fb269f70700 client.5563 path_walk #1/user/nwatkins/in-dir/_partition.lst 2011-11-03 11:14:09.737702 7fb269f70700 client.5563 0 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) user 2011-11-03 11:14:09.737715 7fb269f70700 client.5563 _lookup have dn user mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.737749 7fb269f70700 client.5563 _lookup 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) user = 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) 2011-11-03 11:14:09.737771 7fb269f70700 client.5563 1 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) nwatkins 2011-11-03 11:14:09.737784 7fb269f70700 client.5563 _lookup have dn nwatkins mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.737818 7fb269f70700 client.5563 _lookup 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) nwatkins = 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) 2011-11-03 11:14:09.737840 7fb269f70700 client.5563 2 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) in-dir 2011-11-03 11:14:09.737854 7fb269f70700 client.5563 _lookup have dn in-dir mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.737887 7fb269f70700 client.5563 _lookup 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) in-dir = 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) 2011-11-03 11:14:09.737909 7fb269f70700 client.5563 3 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) _partition.lst 2011-11-03 11:14:09.737939 7fb269f70700 client.5563 _lookup have dn _partition.lst mds.-1 ttl 0.000000 seq 0 2011-11-03 11:14:09.737977 7fb269f70700 client.5563 _lookup 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) _partition.lst = 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.738006 7fb269f70700 client.5563 check_caps on 10000000981.head(ref=1 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) wanted pFscr used - is_delayed=1 2011-11-03 11:14:09.738025 7fb269f70700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.738039 7fb269f70700 inode.get on 0x41bc8660 10000000981.head now 2 2011-11-03 11:14:09.738050 7fb269f70700 client.5563 _create_fh 10000000981 mode 1 2011-11-03 11:14:09.738062 7fb269f70700 client.5563 trim_cache size 12 max 16384 2011-11-03 11:14:09.738078 7fb269f70700 client.5563 open exit(#1/user/nwatkins/in-dir/_partition.lst, 0) = 15 2011-11-03 11:14:09.738091 7fb269f70700 client.5563 describe_layout(15) = 0 2011-11-03 11:14:09.738103 7fb269f70700 client.5563 close enter(15) 2011-11-03 11:14:09.738127 7fb269f70700 client.5563 _release_fh 0x7fb270736750 mode 1 on 10000000981.head(ref=2 cap_refs={} open={1=1} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.738150 7fb269f70700 client.5563 _flush 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.738162 7fb269f70700 client.5563 nothing to flush 2011-11-03 11:14:09.738186 7fb269f70700 client.5563 check_caps on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) wanted - used - is_delayed=0 2011-11-03 11:14:09.738209 7fb269f70700 client.5563 cap_delay_requeue on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.738230 7fb269f70700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:09.738242 7fb269f70700 client.5563 delaying cap release 2011-11-03 11:14:09.738265 7fb269f70700 client.5563 put_inode on 10000000981.head(ref=2 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) 2011-11-03 11:14:09.738280 7fb269f70700 inode.put on 0x41bc8660 10000000981.head now 1 2011-11-03 11:14:09.738292 7fb269f70700 client.5563 close exit(15) 2011-11-03 11:14:09.940014 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:09.940097 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:10.528179 7fb264e5e700 client.5563 renew_caps() 2011-11-03 11:14:10.528243 7fb264e5e700 client.5563 renew_caps requesting from mds.0 2011-11-03 11:14:10.528258 7fb264e5e700 client.5563 renew_caps mds.0 2011-11-03 11:14:10.528325 7fb264e5e700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_renewcaps seq 2) v1 -- ?+0 0x41bca210 2011-11-03 11:14:10.528364 7fb264e5e700 -- 192.168.141.130:0/3011511 submit_message client_session(request_renewcaps seq 2) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:10.528437 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:10.528518 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 18 0x41bca210 client_session(request_renewcaps seq 2) v1 2011-11-03 11:14:10.528547 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 18 0x41bca210 2011-11-03 11:14:10.528568 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x41bca210 2011-11-03 11:14:10.528605 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:10.528622 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:10.528877 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:10.528948 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:10.528975 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:10.529003 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:14:10.529022 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:14:10.529049 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 28 2011-11-03 11:14:10.529072 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:10.529088 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:14:10.529126 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 16 0x41bca210 client_session(renewcaps seq 2) v1 2011-11-03 11:14:10.529152 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:10.529183 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:10.529216 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41bca210 2011-11-03 11:14:10.529253 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 16 ==== client_session(renewcaps seq 2) v1 ==== 28+0+0 (1357429227 0 0) 0x41bca210 con 0x7fb27061cef0 2011-11-03 11:14:10.529273 7fb263e5c700 client.5563 handle_client_session client_session(renewcaps seq 2) v1 2011-11-03 11:14:10.529330 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:10.529403 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 16 2011-11-03 11:14:10.529444 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:14:10.529490 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41bca210 2011-11-03 11:14:10.529522 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:10.529545 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:12.053755 7fb26b376700 -- 192.168.141.130:0/1011511 send_keepalive con 0x7fb2709b88f0, have pipe. 2011-11-03 11:14:12.053858 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:12.053922 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_keepalive 2011-11-03 11:14:12.053957 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:12.053976 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:14:12.523545 7fb26365b700 -- 192.168.141.130:0/3011511 send_keepalive con 0x41cc13a0, have pipe. 2011-11-03 11:14:12.523615 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:12.523647 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_keepalive 2011-11-03 11:14:12.523680 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:12.523698 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:12.723039 7fb268c6b700 client.5562 check_caps on 10000000986.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) wanted Fc used Fc is_delayed=1 2011-11-03 11:14:12.723071 7fb268c6b700 client.5562 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:12.723111 7fb268c6b700 client.5562 send_cap 10000000986.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41875680 0x418771c0) mds.0 seq 1 used Fc want Fc flush - retain pAsxLsxXsxFsxcrwbl held pAsLsXsFscr revoking - dropping - 2011-11-03 11:14:12.723128 7fb268c6b700 client.5562 auth cap, setting max_size = 0 2011-11-03 11:14:12.723160 7fb268c6b700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 10000000986 6761 seq 1 caps=pAsLsXsFscr dirty=- wanted=Fc follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 -- ?+0 0x41bcacb0 2011-11-03 11:14:12.723194 7fb268c6b700 -- 192.168.141.130:0/2011511 submit_message client_caps(update ino 10000000986 6761 seq 1 caps=pAsLsXsFscr dirty=- wanted=Fc follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:12.723229 7fb268c6b700 client.5562 check_caps on 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41879060 0x4188e010) wanted - used - is_delayed=1 2011-11-03 11:14:12.723256 7fb268c6b700 client.5562 cap mds.0 issued pAsLsXsFs implemented pAsLsXsFs revoking - 2011-11-03 11:14:12.723291 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:12.723371 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 23 0x41bcacb0 client_caps(update ino 10000000986 6761 seq 1 caps=pAsLsXsFscr dirty=- wanted=Fc follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 2011-11-03 11:14:12.723408 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 23 0x41bcacb0 2011-11-03 11:14:12.723428 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41bcacb0 2011-11-03 11:14:12.723460 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:12.723477 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:12.940051 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:12.940114 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:15.060189 7fb26cb79700 client.5557 check_caps on 100000000b2.head(ref=1 cap_refs={1024=-3,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) wanted Fc used Fc is_delayed=1 2011-11-03 11:14:15.060218 7fb26cb79700 client.5557 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:15.060252 7fb26cb79700 client.5557 send_cap 100000000b2.head(ref=1 cap_refs={1024=-3,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] parents=0x417e6b80 0x41899360) mds.0 seq 8 used Fc want Fc flush - retain pAsxLsxXsxFsxcrwbl held pAsLsXsFscr revoking - dropping - 2011-11-03 11:14:15.060267 7fb26cb79700 client.5557 auth cap, setting max_size = 0 2011-11-03 11:14:15.060293 7fb26cb79700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 100000000b2 6752 seq 8 caps=pAsLsXsFscr dirty=- wanted=Fc follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 -- ?+0 0x41bcacb0 2011-11-03 11:14:15.060332 7fb26cb79700 -- 192.168.141.130:0/1011511 submit_message client_caps(update ino 100000000b2 6752 seq 8 caps=pAsLsXsFscr dirty=- wanted=Fc follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:15.060394 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:15.060477 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 19 0x41bcacb0 client_caps(update ino 100000000b2 6752 seq 8 caps=pAsLsXsFscr dirty=- wanted=Fc follows 0 size 106/0 ts 1 mtime 2011-11-03 11:14:05.788394) v1 2011-11-03 11:14:15.060513 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 19 0x41bcacb0 2011-11-03 11:14:15.060534 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x41bcacb0 2011-11-03 11:14:15.060567 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:15.060585 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:15.279986 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:15.280088 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:15.528587 7fb264e5e700 client.5563 check_caps on 10000000986.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) wanted Fc used Fc is_delayed=1 2011-11-03 11:14:15.528655 7fb264e5e700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:15.528694 7fb264e5e700 client.5563 send_cap 10000000986.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] parents=0x41cc6c30 0x41bc45b0) mds.0 seq 1 used Fc want Fc flush - retain pAsxLsxXsxFsxcrwbl held pAsLsXsFscr revoking - dropping - 2011-11-03 11:14:15.528711 7fb264e5e700 client.5563 auth cap, setting max_size = 0 2011-11-03 11:14:15.528748 7fb264e5e700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 10000000986 6774 seq 1 caps=pAsLsXsFscr dirty=- wanted=Fc follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 -- ?+0 0x41bcacb0 2011-11-03 11:14:15.528793 7fb264e5e700 -- 192.168.141.130:0/3011511 submit_message client_caps(update ino 10000000986 6774 seq 1 caps=pAsLsXsFscr dirty=- wanted=Fc follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:15.528831 7fb264e5e700 client.5563 check_caps on 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) wanted - used - is_delayed=1 2011-11-03 11:14:15.528858 7fb264e5e700 client.5563 cap mds.0 issued pAsLsXsFscr implemented pAsLsXsFscr revoking - 2011-11-03 11:14:15.528894 7fb264e5e700 client.5563 send_cap 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] parents=0x41bc5dd0 0x41bc8660) mds.0 seq 1 used - want - flush - retain pAsLsXsFs held pAsLsXsFscr revoking - dropping Fcr 2011-11-03 11:14:15.528913 7fb264e5e700 client.5563 auth cap, setting max_size = 0 2011-11-03 11:14:15.528941 7fb264e5e700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_caps(update ino 10000000981 6778 seq 1 caps=pAsLsXsFs dirty=- wanted=- follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 -- ?+0 0x41debb90 2011-11-03 11:14:15.528995 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:15.529094 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 19 0x41bcacb0 client_caps(update ino 10000000986 6774 seq 1 caps=pAsLsXsFscr dirty=- wanted=Fc follows 0 size 17110/0 ts 1 mtime 2011-11-03 11:14:04.597264) v1 2011-11-03 11:14:15.529131 7fb264e5e700 -- 192.168.141.130:0/3011511 submit_message client_caps(update ino 10000000981 6778 seq 1 caps=pAsLsXsFs dirty=- wanted=- follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:15.529162 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 19 0x41bcacb0 2011-11-03 11:14:15.529188 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x41bcacb0 2011-11-03 11:14:15.529232 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:15.529293 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 20 0x41debb90 client_caps(update ino 10000000981 6778 seq 1 caps=pAsLsXsFs dirty=- wanted=- follows 0 size 129/0 ts 1 mtime 2011-11-03 11:14:01.196477) v1 2011-11-03 11:14:15.529315 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 20 0x41debb90 2011-11-03 11:14:15.529332 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x41debb90 2011-11-03 11:14:15.529362 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:15.529379 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:15.740726 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:15.740804 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:15.740831 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:15.740856 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:19.719040 7fb267468700 -- 192.168.141.130:0/2011511 send_keepalive con 0x41c05490, have pipe. 2011-11-03 11:14:19.719151 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:19.719208 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).write_keepalive 2011-11-03 11:14:19.719238 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:19.719255 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer sleeping 2011-11-03 11:14:20.060521 7fb26cb79700 client.5557 renew_caps() 2011-11-03 11:14:20.060589 7fb26cb79700 client.5557 renew_caps requesting from mds.0 2011-11-03 11:14:20.060603 7fb26cb79700 client.5557 renew_caps mds.0 2011-11-03 11:14:20.060636 7fb26cb79700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_renewcaps seq 4) v1 -- ?+0 0x41bca770 2011-11-03 11:14:20.060658 7fb26cb79700 -- 192.168.141.130:0/1011511 submit_message client_session(request_renewcaps seq 4) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:20.060703 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:20.060736 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 20 0x41bca770 client_session(request_renewcaps seq 4) v1 2011-11-03 11:14:20.060768 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 20 0x41bca770 2011-11-03 11:14:20.060789 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x41bca770 2011-11-03 11:14:20.060821 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:20.060838 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:20.061047 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:20.061108 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:20.061132 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:20.061163 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:14:20.061182 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:14:20.061208 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 28 2011-11-03 11:14:20.061232 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:20.061249 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:14:20.061284 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 20 0x41bca770 client_session(renewcaps seq 4) v1 2011-11-03 11:14:20.061314 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:20.061341 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:20.061373 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x41bca770 2011-11-03 11:14:20.061404 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 20 ==== client_session(renewcaps seq 4) v1 ==== 28+0+0 (3279728092 0 0) 0x41bca770 con 0x7fb2709b9a40 2011-11-03 11:14:20.061426 7fb26bb77700 client.5557 handle_client_session client_session(renewcaps seq 4) v1 2011-11-03 11:14:20.061455 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:20.061483 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 20 2011-11-03 11:14:20.061509 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:20.061530 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:20.061560 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:14:20.061580 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x41bca770 2011-11-03 11:14:22.053920 7fb26b376700 -- 192.168.141.130:0/1011511 send_keepalive con 0x7fb2709b88f0, have pipe. 2011-11-03 11:14:22.053989 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:22.054047 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).write_keepalive 2011-11-03 11:14:22.054087 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:22.054105 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer sleeping 2011-11-03 11:14:22.523705 7fb26365b700 -- 192.168.141.130:0/3011511 send_keepalive con 0x41cc13a0, have pipe. 2011-11-03 11:14:22.523811 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:22.523836 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).write_keepalive 2011-11-03 11:14:22.523864 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer: state = 2 policy.server=0 2011-11-03 11:14:22.523881 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer sleeping 2011-11-03 11:14:26.688100 7fb261b4e700 client.5557 unmounting 2011-11-03 11:14:26.688346 7fb261b4e700 client.5557 put_inode on 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) 2011-11-03 11:14:26.688365 7fb261b4e700 inode.put on 0x41deb720 1.head now 2 2011-11-03 11:14:26.688381 7fb261b4e700 client.5557 _ll_drop_pins 2011-11-03 11:14:26.688398 7fb261b4e700 client.5557 _ll_put 0x41deb720 1 1 -> 0 2011-11-03 11:14:26.688419 7fb261b4e700 client.5557 put_inode on 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) 2011-11-03 11:14:26.688432 7fb261b4e700 inode.put on 0x41deb720 1.head now 1 2011-11-03 11:14:26.688448 7fb261b4e700 client.5557 trim_cache size 6 max 0 2011-11-03 11:14:26.688465 7fb261b4e700 client.5557 trim_dentry unlinking dn jobToken in dir 100000000b1 2011-11-03 11:14:26.688484 7fb261b4e700 client.5557 unlink dir 0x41898600 'jobToken' dn 0x417e6b80 inode 0x41899360 2011-11-03 11:14:26.688500 7fb261b4e700 client.5557 unlink inode 0x41899360 parents now 2011-11-03 11:14:26.688529 7fb261b4e700 client.5557 put_inode on 100000000b2.head(ref=1 cap_refs={1024=-3,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] 0x41899360) 2011-11-03 11:14:26.688542 7fb261b4e700 inode.put on 0x41899360 100000000b2.head now 0 2011-11-03 11:14:26.688571 7fb261b4e700 client.5557 remove_cap mds.0 on 100000000b2.head(ref=0 cap_refs={1024=-3,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] 0x41899360) 2011-11-03 11:14:26.688597 7fb261b4e700 client.5557 remove_cap last one, closing snaprealm 0x41c33f90 2011-11-03 11:14:26.688619 7fb261b4e700 client.5557 put_inode deleting 100000000b2.head(ref=0 cap_refs={1024=-3,2048=0,8192=0} open={1=0} mode=100777 size=106 mtime=2011-11-03 11:14:05.788394 caps=- objectset[100000000b2 ts 0/0 objects 1 dirty_or_tx 0] 0x41899360) 2011-11-03 11:14:26.688680 7fb261b4e700 client.5557 close_dir dir 0x417e6b00 on 0x41898600 2011-11-03 11:14:26.688707 7fb261b4e700 client.5557 put_inode on 100000000b1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) parents=0x41c358a0 0x41898600) 2011-11-03 11:14:26.688721 7fb261b4e700 inode.put on 0x41898600 100000000b1.head now 1 2011-11-03 11:14:26.688734 7fb261b4e700 client.5557 trim_dentry unlinking dn job_201111031113_0001 in dir 100000000af 2011-11-03 11:14:26.688747 7fb261b4e700 client.5557 unlink dir 0x41897840 'job_201111031113_0001' dn 0x41c358a0 inode 0x41898600 2011-11-03 11:14:26.688759 7fb261b4e700 client.5557 unlink inode 0x41898600 parents now 2011-11-03 11:14:26.688779 7fb261b4e700 client.5557 put_inode on 100000000b1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) 0x41898600) 2011-11-03 11:14:26.688792 7fb261b4e700 inode.put on 0x41898600 100000000b1.head now 0 2011-11-03 11:14:26.688812 7fb261b4e700 client.5557 remove_cap mds.0 on 100000000b1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=pAsLs(0=pAsLs) 0x41898600) 2011-11-03 11:14:26.688836 7fb261b4e700 client.5557 remove_cap last one, closing snaprealm 0x41c33f90 2011-11-03 11:14:26.688869 7fb261b4e700 client.5557 put_inode deleting 100000000b1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.785578 caps=- 0x41898600) 2011-11-03 11:14:26.688883 7fb261b4e700 client.5557 close_dir dir 0x417e6980 on 0x41897840 2011-11-03 11:14:26.688907 7fb261b4e700 client.5557 put_inode on 100000000af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) parents=0x417e5530 0x41897840) 2011-11-03 11:14:26.688919 7fb261b4e700 inode.put on 0x41897840 100000000af.head now 1 2011-11-03 11:14:26.688933 7fb261b4e700 client.5557 trim_dentry unlinking dn system in dir 1000000005e 2011-11-03 11:14:26.688945 7fb261b4e700 client.5557 unlink dir 0x417e6510 'system' dn 0x417e5530 inode 0x41897840 2011-11-03 11:14:26.688957 7fb261b4e700 client.5557 unlink inode 0x41897840 parents now 2011-11-03 11:14:26.688978 7fb261b4e700 client.5557 put_inode on 100000000af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) 0x41897840) 2011-11-03 11:14:26.688991 7fb261b4e700 inode.put on 0x41897840 100000000af.head now 0 2011-11-03 11:14:26.689012 7fb261b4e700 client.5557 remove_cap mds.0 on 100000000af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=pAsLsXs(0=pAsLsXs) 0x41897840) 2011-11-03 11:14:26.689026 7fb261b4e700 client.5557 remove_cap last one, closing snaprealm 0x41c33f90 2011-11-03 11:14:26.689042 7fb261b4e700 client.5557 put_inode deleting 100000000af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:05.781412 caps=- 0x41897840) 2011-11-03 11:14:26.689055 7fb261b4e700 client.5557 close_dir dir 0x417e54b0 on 0x417e6510 2011-11-03 11:14:26.689078 7fb261b4e700 client.5557 put_inode on 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x417e56e0 0x417e6510) 2011-11-03 11:14:26.689091 7fb261b4e700 inode.put on 0x417e6510 1000000005e.head now 1 2011-11-03 11:14:26.689103 7fb261b4e700 client.5557 trim_dentry unlinking dn mapred in dir 1000000005d 2011-11-03 11:14:26.689116 7fb261b4e700 client.5557 unlink dir 0x41c350b0 'mapred' dn 0x417e56e0 inode 0x417e6510 2011-11-03 11:14:26.689128 7fb261b4e700 client.5557 unlink inode 0x417e6510 parents now 2011-11-03 11:14:26.689149 7fb261b4e700 client.5557 put_inode on 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) 0x417e6510) 2011-11-03 11:14:26.689162 7fb261b4e700 inode.put on 0x417e6510 1000000005e.head now 0 2011-11-03 11:14:26.689182 7fb261b4e700 client.5557 remove_cap mds.0 on 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) 0x417e6510) 2011-11-03 11:14:26.689195 7fb261b4e700 client.5557 remove_cap last one, closing snaprealm 0x41c33f90 2011-11-03 11:14:26.689211 7fb261b4e700 client.5557 put_inode deleting 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=- 0x417e6510) 2011-11-03 11:14:26.689224 7fb261b4e700 client.5557 close_dir dir 0x41c35770 on 0x41c350b0 2011-11-03 11:14:26.689249 7fb261b4e700 client.5557 put_inode on 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417e6170 0x41c350b0) 2011-11-03 11:14:26.689262 7fb261b4e700 inode.put on 0x41c350b0 1000000005d.head now 1 2011-11-03 11:14:26.689275 7fb261b4e700 client.5557 trim_dentry unlinking dn hadoop-nwatkins in dir 1000000005c 2011-11-03 11:14:26.689287 7fb261b4e700 client.5557 unlink dir 0x41c359c0 'hadoop-nwatkins' dn 0x417e6170 inode 0x41c350b0 2011-11-03 11:14:26.689300 7fb261b4e700 client.5557 unlink inode 0x41c350b0 parents now 2011-11-03 11:14:26.689321 7fb261b4e700 client.5557 put_inode on 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41c350b0) 2011-11-03 11:14:26.689340 7fb261b4e700 inode.put on 0x41c350b0 1000000005d.head now 0 2011-11-03 11:14:26.689376 7fb261b4e700 client.5557 remove_cap mds.0 on 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41c350b0) 2011-11-03 11:14:26.689392 7fb261b4e700 client.5557 remove_cap last one, closing snaprealm 0x41c33f90 2011-11-03 11:14:26.689408 7fb261b4e700 client.5557 put_inode deleting 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=- 0x41c350b0) 2011-11-03 11:14:26.689423 7fb261b4e700 client.5557 close_dir dir 0x417e5ae0 on 0x41c359c0 2011-11-03 11:14:26.689448 7fb261b4e700 client.5557 put_inode on 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c35590 0x41c359c0) 2011-11-03 11:14:26.689463 7fb261b4e700 inode.put on 0x41c359c0 1000000005c.head now 1 2011-11-03 11:14:26.689478 7fb261b4e700 client.5557 trim_dentry unlinking dn tmp in dir 1 2011-11-03 11:14:26.689492 7fb261b4e700 client.5557 unlink dir 0x41deb720 'tmp' dn 0x41c35590 inode 0x41c359c0 2011-11-03 11:14:26.689506 7fb261b4e700 client.5557 unlink inode 0x41c359c0 parents now 2011-11-03 11:14:26.689532 7fb261b4e700 client.5557 put_inode on 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41c359c0) 2011-11-03 11:14:26.689547 7fb261b4e700 inode.put on 0x41c359c0 1000000005c.head now 0 2011-11-03 11:14:26.689571 7fb261b4e700 client.5557 remove_cap mds.0 on 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41c359c0) 2011-11-03 11:14:26.689585 7fb261b4e700 client.5557 remove_cap last one, closing snaprealm 0x41c33f90 2011-11-03 11:14:26.689600 7fb261b4e700 client.5557 put_inode deleting 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=- 0x41c359c0) 2011-11-03 11:14:26.689614 7fb261b4e700 client.5557 close_dir dir 0x417e4df0 on 0x41deb720 2011-11-03 11:14:26.689638 7fb261b4e700 client.5557 put_inode on 1.head(ref=1 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) 2011-11-03 11:14:26.689650 7fb261b4e700 inode.put on 0x41deb720 1.head now 0 2011-11-03 11:14:26.689671 7fb261b4e700 client.5557 remove_cap mds.0 on 1.head(ref=0 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41deb720) 2011-11-03 11:14:26.689684 7fb261b4e700 client.5557 remove_cap last one, closing snaprealm 0x41c33f90 2011-11-03 11:14:26.689699 7fb261b4e700 client.5557 put_inode deleting 1.head(ref=0 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=- 0x41deb720) 2011-11-03 11:14:26.689717 7fb261b4e700 client.5557 flush_caps 2011-11-03 11:14:26.689731 7fb261b4e700 client.5557 wait_sync_caps want 0 (last is 0, 0 total flushing) 2011-11-03 11:14:26.689744 7fb261b4e700 client.5557 sending client_session close to mds.0 seq 4 2011-11-03 11:14:26.689771 7fb261b4e700 -- 192.168.141.130:0/1011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_close seq 4) v1 -- ?+0 0x417e56e0 2011-11-03 11:14:26.689792 7fb261b4e700 -- 192.168.141.130:0/1011511 submit_message client_session(request_close seq 4) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:26.689818 7fb261b4e700 client.5557 waiting for 1 mds sessions to close 2011-11-03 11:14:26.689874 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:26.689938 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer encoding 21 0x417e56e0 client_session(request_close seq 4) v1 2011-11-03 11:14:26.689985 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sending 21 0x417e56e0 2011-11-03 11:14:26.690032 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_message 0x417e56e0 2011-11-03 11:14:26.690095 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:26.690114 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:26.899985 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got ACK 2011-11-03 11:14:26.900063 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:26.918015 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got MSG 2011-11-03 11:14:26.918048 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:14:26.918067 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:14:26.918094 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got front 28 2011-11-03 11:14:26.918117 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).aborted = 0 2011-11-03 11:14:26.918134 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:14:26.918169 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader got message 21 0x417e56a0 client_session(close) v1 2011-11-03 11:14:26.918194 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:26.918223 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader reading tag... 2011-11-03 11:14:26.918261 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_entry pipe 0x7fb2709b8c40 dequeued 0x417e56a0 2011-11-03 11:14:26.918308 7fb26bb77700 -- 192.168.141.130:0/1011511 <== mds.0 192.168.141.123:6802/19591 21 ==== client_session(close) v1 ==== 28+0+0 (3822250925 0 0) 0x417e56a0 con 0x7fb2709b9a40 2011-11-03 11:14:26.918332 7fb26bb77700 client.5557 handle_client_session client_session(close) v1 2011-11-03 11:14:26.918366 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:26.918394 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).write_ack 21 2011-11-03 11:14:26.918417 7fb26bb77700 client.5557 kick_requests for mds.0 2011-11-03 11:14:26.918452 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:26.918477 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer sleeping 2011-11-03 11:14:26.918500 7fb26bb77700 client.5557 unmounting: trim pass, size was 0+0 2011-11-03 11:14:26.918519 7fb26bb77700 client.5557 trim_cache size 0 max 0 2011-11-03 11:14:26.918532 7fb26bb77700 client.5557 unmounting: trim pass, cache now empty, waking unmount() 2011-11-03 11:14:26.918553 7fb26bb77700 -- 192.168.141.130:0/1011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:14:26.918581 7fb26bb77700 -- 192.168.141.130:0/1011511 done calling dispatch on 0x417e56a0 2011-11-03 11:14:26.918631 7fb261b4e700 client.5557 unmounted. 2011-11-03 11:14:26.918702 7fb261b4e700 -- 192.168.141.130:0/1011511 mark_down 0x417e6380 -- 0x41899050 2011-11-03 11:14:26.918727 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).unregister_pipe 2011-11-03 11:14:26.918752 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).stop 2011-11-03 11:14:26.918825 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader couldn't read tag, Success 2011-11-03 11:14:26.918859 7fb261b4e700 client.5557 shutdown 2011-11-03 11:14:26.918899 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).fault 0: Success 2011-11-03 11:14:26.918921 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).fault already closed|closing 2011-11-03 11:14:26.918945 7fb269d6e700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).reader done 2011-11-03 11:14:26.919010 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).writer finishing 2011-11-03 11:14:26.919041 7fb269e6f700 -- 192.168.141.130:0/1011511 queue_reap 0x41899050 2011-11-03 11:14:26.919064 7fb269e6f700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).writer done 2011-11-03 11:14:26.919090 7fb261b4e700 -- 192.168.141.130:0/1011511 shutdown 192.168.141.130:0/1011511 2011-11-03 11:14:26.919116 7fb261b4e700 -- 192.168.141.130:0/1011511 shutdown i am not dispatch, setting stop flag and joining thread. 2011-11-03 11:14:26.919151 7fb26d37a700 -- 192.168.141.130:0/1011511 reaper 2011-11-03 11:14:26.919179 7fb26d37a700 -- 192.168.141.130:0/1011511 reaper reaping pipe 0x41899050 192.168.141.127:6800/3516 2011-11-03 11:14:26.919204 7fb26d37a700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).discard_queue 2011-11-03 11:14:26.919228 7fb261b4e700 client.5557 trim_cache size 0 max 0 2011-11-03 11:14:26.919261 7fb26d37a700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1). dequeued pipe 2011-11-03 11:14:26.919288 7fb26d37a700 -- 192.168.141.130:0/1011511 >> 192.168.141.127:6800/3516 pipe(0x41899050 sd=84 pgs=716 cs=1 l=1).unregister_pipe - not registered 2011-11-03 11:14:26.919326 7fb26d37a700 -- 192.168.141.130:0/1011511 reaper reaped pipe 0x41899050 192.168.141.127:6800/3516 2011-11-03 11:14:26.919352 7fb26d37a700 -- 192.168.141.130:0/1011511 reaper deleted pipe 0x41899050 2011-11-03 11:14:26.919367 7fb26d37a700 -- 192.168.141.130:0/1011511 reaper done 2011-11-03 11:14:26.919404 7fb261b4e700 -- 192.168.141.130:0/1011511 wait: everything stopped 2011-11-03 11:14:26.919424 7fb261b4e700 -- 192.168.141.130:0/1011511 wait: stopping reaper thread 2011-11-03 11:14:26.919450 7fb26d37a700 -- 192.168.141.130:0/1011511 reaper_entry done 2011-11-03 11:14:26.919551 7fb261b4e700 -- 192.168.141.130:0/1011511 wait: stopped reaper thread 2011-11-03 11:14:26.919571 7fb261b4e700 -- 192.168.141.130:0/1011511 wait: closing pipes 2011-11-03 11:14:26.919590 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).unregister_pipe 2011-11-03 11:14:26.919607 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).stop 2011-11-03 11:14:26.919648 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).unregister_pipe 2011-11-03 11:14:26.919671 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).stop 2011-11-03 11:14:26.919707 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer finishing 2011-11-03 11:14:26.919763 7fb26a374700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).writer done 2011-11-03 11:14:26.919819 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader couldn't read tag, Success 2011-11-03 11:14:26.919873 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).fault 0: Success 2011-11-03 11:14:26.919895 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).fault already closed|closing 2011-11-03 11:14:26.919919 7fb26a071700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).reader done 2011-11-03 11:14:26.919959 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader couldn't read tag, Success 2011-11-03 11:14:26.919990 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).fault 0: Success 2011-11-03 11:14:26.920012 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).fault already closed|closing 2011-11-03 11:14:26.920033 7fb26a273700 -- 192.168.141.130:0/1011511 queue_reap 0x7fb2709b8680 2011-11-03 11:14:26.920058 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper 2011-11-03 11:14:26.920082 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper done 2011-11-03 11:14:26.920102 7fb261b4e700 -- 192.168.141.130:0/1011511 wait: waiting for pipes 0x7fb2709b8680,0x7fb2709b8c40 to close 2011-11-03 11:14:26.920131 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer finishing 2011-11-03 11:14:26.920160 7fb26a172700 -- 192.168.141.130:0/1011511 queue_reap 0x7fb2709b8c40 2011-11-03 11:14:26.920184 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper 2011-11-03 11:14:26.920209 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper reaping pipe 0x7fb2709b8680 192.168.141.123:6789/0 2011-11-03 11:14:26.920232 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).discard_queue 2011-11-03 11:14:26.920256 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1). dequeued pipe 2011-11-03 11:14:26.920279 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).unregister_pipe - not registered 2011-11-03 11:14:26.920306 7fb26a273700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6789/0 pipe(0x7fb2709b8680 sd=80 pgs=375 cs=1 l=1).reader done 2011-11-03 11:14:26.920367 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper reaped pipe 0x7fb2709b8680 192.168.141.123:6789/0 2011-11-03 11:14:26.920388 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper deleted pipe 0x7fb2709b8680 2011-11-03 11:14:26.920402 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper done 2011-11-03 11:14:26.920435 7fb26a172700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).writer done 2011-11-03 11:14:26.920464 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper 2011-11-03 11:14:26.920485 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper reaping pipe 0x7fb2709b8c40 192.168.141.123:6802/19591 2011-11-03 11:14:26.920502 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).discard_queue 2011-11-03 11:14:26.920520 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0). dequeued pipe 2011-11-03 11:14:26.920537 7fb261b4e700 -- 192.168.141.130:0/1011511 >> 192.168.141.123:6802/19591 pipe(0x7fb2709b8c40 sd=81 pgs=1174 cs=1 l=0).unregister_pipe - not registered 2011-11-03 11:14:26.920578 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper reaped pipe 0x7fb2709b8c40 192.168.141.123:6802/19591 2011-11-03 11:14:26.920607 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper deleted pipe 0x7fb2709b8c40 2011-11-03 11:14:26.920621 7fb261b4e700 -- 192.168.141.130:0/1011511 reaper done 2011-11-03 11:14:26.920635 7fb261b4e700 -- 192.168.141.130:0/1011511 wait: done. 2011-11-03 11:14:26.920648 7fb261b4e700 -- 192.168.141.130:0/1011511 shutdown complete. 2011-11-03 11:14:26.920849 7fb261b4e700 client.5562 unmounting 2011-11-03 11:14:26.920888 7fb261b4e700 client.5562 put_inode on 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) 2011-11-03 11:14:26.920901 7fb261b4e700 inode.put on 0x41b66000 1.head now 2 2011-11-03 11:14:26.920914 7fb261b4e700 client.5562 _ll_drop_pins 2011-11-03 11:14:26.920928 7fb261b4e700 client.5562 _ll_put 0x41b66000 1 1 -> 0 2011-11-03 11:14:26.920949 7fb261b4e700 client.5562 put_inode on 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) 2011-11-03 11:14:26.920961 7fb261b4e700 inode.put on 0x41b66000 1.head now 1 2011-11-03 11:14:26.920976 7fb261b4e700 client.5562 trim_cache size 12 max 0 2011-11-03 11:14:26.920990 7fb261b4e700 client.5562 trim_dentry unlinking dn job.xml in dir 10000000982 2011-11-03 11:14:26.921003 7fb261b4e700 client.5562 unlink dir 0x418764e0 'job.xml' dn 0x41875680 inode 0x418771c0 2011-11-03 11:14:26.921016 7fb261b4e700 client.5562 unlink inode 0x418771c0 parents now 2011-11-03 11:14:26.921042 7fb261b4e700 client.5562 put_inode on 10000000986.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] 0x418771c0) 2011-11-03 11:14:26.921055 7fb261b4e700 inode.put on 0x418771c0 10000000986.head now 0 2011-11-03 11:14:26.921080 7fb261b4e700 client.5562 remove_cap mds.0 on 10000000986.head(ref=0 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] 0x418771c0) 2011-11-03 11:14:26.921102 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.921121 7fb261b4e700 client.5562 put_inode deleting 10000000986.head(ref=0 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=- objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] 0x418771c0) 2011-11-03 11:14:26.921148 7fb261b4e700 client.5562 close_dir dir 0x418762a0 on 0x418764e0 2011-11-03 11:14:26.921173 7fb261b4e700 client.5562 put_inode on 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x418755c0 0x418764e0) 2011-11-03 11:14:26.921185 7fb261b4e700 inode.put on 0x418764e0 10000000982.head now 1 2011-11-03 11:14:26.921198 7fb261b4e700 client.5562 trim_dentry unlinking dn job_201111031113_0001 in dir 100000008af 2011-11-03 11:14:26.921210 7fb261b4e700 client.5562 unlink dir 0x41875740 'job_201111031113_0001' dn 0x418755c0 inode 0x418764e0 2011-11-03 11:14:26.921222 7fb261b4e700 client.5562 unlink inode 0x418764e0 parents now 2011-11-03 11:14:26.921242 7fb261b4e700 client.5562 put_inode on 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) 0x418764e0) 2011-11-03 11:14:26.921255 7fb261b4e700 inode.put on 0x418764e0 10000000982.head now 0 2011-11-03 11:14:26.921278 7fb261b4e700 client.5562 remove_cap mds.0 on 10000000982.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) 0x418764e0) 2011-11-03 11:14:26.921292 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.921307 7fb261b4e700 client.5562 put_inode deleting 10000000982.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=- 0x418764e0) 2011-11-03 11:14:26.921327 7fb261b4e700 client.5562 close_dir dir 0x41875540 on 0x41875740 2011-11-03 11:14:26.921359 7fb261b4e700 client.5562 put_inode on 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) parents=0x41b66620 0x41875740) 2011-11-03 11:14:26.921372 7fb261b4e700 inode.put on 0x41875740 100000008af.head now 1 2011-11-03 11:14:26.921385 7fb261b4e700 client.5562 trim_dentry unlinking dn .staging in dir 100000008ae 2011-11-03 11:14:26.921397 7fb261b4e700 client.5562 unlink dir 0x41874a20 '.staging' dn 0x41b66620 inode 0x41875740 2011-11-03 11:14:26.921409 7fb261b4e700 client.5562 unlink inode 0x41875740 parents now 2011-11-03 11:14:26.921429 7fb261b4e700 client.5562 put_inode on 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) 0x41875740) 2011-11-03 11:14:26.921442 7fb261b4e700 inode.put on 0x41875740 100000008af.head now 0 2011-11-03 11:14:26.921462 7fb261b4e700 client.5562 remove_cap mds.0 on 100000008af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXs(0=pAsLsXs) 0x41875740) 2011-11-03 11:14:26.921475 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.921490 7fb261b4e700 client.5562 put_inode deleting 100000008af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=- 0x41875740) 2011-11-03 11:14:26.921503 7fb261b4e700 client.5562 close_dir dir 0x41874800 on 0x41874a20 2011-11-03 11:14:26.921527 7fb261b4e700 client.5562 put_inode on 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41b66440 0x41874a20) 2011-11-03 11:14:26.921540 7fb261b4e700 inode.put on 0x41874a20 100000008ae.head now 1 2011-11-03 11:14:26.921553 7fb261b4e700 client.5562 trim_dentry unlinking dn nwatkins in dir 100000008ad 2011-11-03 11:14:26.921565 7fb261b4e700 client.5562 unlink dir 0x41873ca0 'nwatkins' dn 0x41b66440 inode 0x41874a20 2011-11-03 11:14:26.921577 7fb261b4e700 client.5562 unlink inode 0x41874a20 parents now 2011-11-03 11:14:26.921598 7fb261b4e700 client.5562 put_inode on 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41874a20) 2011-11-03 11:14:26.921610 7fb261b4e700 inode.put on 0x41874a20 100000008ae.head now 0 2011-11-03 11:14:26.921631 7fb261b4e700 client.5562 remove_cap mds.0 on 100000008ae.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41874a20) 2011-11-03 11:14:26.921643 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.921658 7fb261b4e700 client.5562 put_inode deleting 100000008ae.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=- 0x41874a20) 2011-11-03 11:14:26.921672 7fb261b4e700 client.5562 close_dir dir 0x41b665a0 on 0x41873ca0 2011-11-03 11:14:26.921696 7fb261b4e700 client.5562 put_inode on 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c01160 0x41873ca0) 2011-11-03 11:14:26.921708 7fb261b4e700 inode.put on 0x41873ca0 100000008ad.head now 1 2011-11-03 11:14:26.921720 7fb261b4e700 client.5562 trim_dentry unlinking dn staging in dir 1000000005e 2011-11-03 11:14:26.921733 7fb261b4e700 client.5562 unlink dir 0x41c04380 'staging' dn 0x41c01160 inode 0x41873ca0 2011-11-03 11:14:26.921745 7fb261b4e700 client.5562 unlink inode 0x41873ca0 parents now 2011-11-03 11:14:26.921765 7fb261b4e700 client.5562 put_inode on 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41873ca0) 2011-11-03 11:14:26.921778 7fb261b4e700 inode.put on 0x41873ca0 100000008ad.head now 0 2011-11-03 11:14:26.921799 7fb261b4e700 client.5562 remove_cap mds.0 on 100000008ad.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41873ca0) 2011-11-03 11:14:26.921824 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.921840 7fb261b4e700 client.5562 put_inode deleting 100000008ad.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=- 0x41873ca0) 2011-11-03 11:14:26.921854 7fb261b4e700 client.5562 close_dir dir 0x41b66ef0 on 0x41c04380 2011-11-03 11:14:26.921876 7fb261b4e700 client.5562 put_inode on 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41c047c0 0x41c04380) 2011-11-03 11:14:26.921889 7fb261b4e700 inode.put on 0x41c04380 1000000005e.head now 1 2011-11-03 11:14:26.921901 7fb261b4e700 client.5562 trim_dentry unlinking dn mapred in dir 1000000005d 2011-11-03 11:14:26.921914 7fb261b4e700 client.5562 unlink dir 0x41c03660 'mapred' dn 0x41c047c0 inode 0x41c04380 2011-11-03 11:14:26.921925 7fb261b4e700 client.5562 unlink inode 0x41c04380 parents now 2011-11-03 11:14:26.921946 7fb261b4e700 client.5562 put_inode on 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) 0x41c04380) 2011-11-03 11:14:26.921958 7fb261b4e700 inode.put on 0x41c04380 1000000005e.head now 0 2011-11-03 11:14:26.921978 7fb261b4e700 client.5562 remove_cap mds.0 on 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) 0x41c04380) 2011-11-03 11:14:26.921991 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.922022 7fb261b4e700 client.5562 put_inode deleting 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=- 0x41c04380) 2011-11-03 11:14:26.922035 7fb261b4e700 client.5562 close_dir dir 0x41c02ea0 on 0x41c03660 2011-11-03 11:14:26.922059 7fb261b4e700 client.5562 put_inode on 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41899a00 0x41c03660) 2011-11-03 11:14:26.922071 7fb261b4e700 inode.put on 0x41c03660 1000000005d.head now 1 2011-11-03 11:14:26.922084 7fb261b4e700 client.5562 trim_dentry unlinking dn hadoop-nwatkins in dir 1000000005c 2011-11-03 11:14:26.922096 7fb261b4e700 client.5562 unlink dir 0x41b66970 'hadoop-nwatkins' dn 0x41899a00 inode 0x41c03660 2011-11-03 11:14:26.922108 7fb261b4e700 client.5562 unlink inode 0x41c03660 parents now 2011-11-03 11:14:26.922128 7fb261b4e700 client.5562 put_inode on 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41c03660) 2011-11-03 11:14:26.922141 7fb261b4e700 inode.put on 0x41c03660 1000000005d.head now 0 2011-11-03 11:14:26.922161 7fb261b4e700 client.5562 remove_cap mds.0 on 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41c03660) 2011-11-03 11:14:26.922174 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.922189 7fb261b4e700 client.5562 put_inode deleting 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=- 0x41c03660) 2011-11-03 11:14:26.922202 7fb261b4e700 client.5562 close_dir dir 0x41b66850 on 0x41b66970 2011-11-03 11:14:26.922226 7fb261b4e700 client.5562 put_inode on 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41c00420 0x41b66970) 2011-11-03 11:14:26.922238 7fb261b4e700 inode.put on 0x41b66970 1000000005c.head now 1 2011-11-03 11:14:26.922250 7fb261b4e700 client.5562 trim_dentry unlinking dn tmp in dir 1 2011-11-03 11:14:26.922262 7fb261b4e700 client.5562 unlink dir 0x41b66000 'tmp' dn 0x41c00420 inode 0x41b66970 2011-11-03 11:14:26.922274 7fb261b4e700 client.5562 unlink inode 0x41b66970 parents now 2011-11-03 11:14:26.922295 7fb261b4e700 client.5562 put_inode on 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66970) 2011-11-03 11:14:26.922320 7fb261b4e700 inode.put on 0x41b66970 1000000005c.head now 0 2011-11-03 11:14:26.922342 7fb261b4e700 client.5562 remove_cap mds.0 on 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66970) 2011-11-03 11:14:26.922354 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.922369 7fb261b4e700 client.5562 put_inode deleting 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=- 0x41b66970) 2011-11-03 11:14:26.922384 7fb261b4e700 client.5562 trim_dentry unlinking dn _partition.lst in dir 10000000ce2 2011-11-03 11:14:26.922397 7fb261b4e700 client.5562 unlink dir 0x41878610 '_partition.lst' dn 0x41879060 inode 0x4188e010 2011-11-03 11:14:26.922410 7fb261b4e700 client.5562 unlink inode 0x4188e010 parents now 2011-11-03 11:14:26.922433 7fb261b4e700 client.5562 put_inode on 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x4188e010) 2011-11-03 11:14:26.922445 7fb261b4e700 inode.put on 0x4188e010 10000000981.head now 0 2011-11-03 11:14:26.922467 7fb261b4e700 client.5562 remove_cap mds.0 on 10000000981.head(ref=0 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x4188e010) 2011-11-03 11:14:26.922482 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.922499 7fb261b4e700 client.5562 put_inode deleting 10000000981.head(ref=0 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=- objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x4188e010) 2011-11-03 11:14:26.922513 7fb261b4e700 client.5562 close_dir dir 0x418790f0 on 0x41878610 2011-11-03 11:14:26.922536 7fb261b4e700 client.5562 put_inode on 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879240 0x41878610) 2011-11-03 11:14:26.922549 7fb261b4e700 inode.put on 0x41878610 10000000ce2.head now 1 2011-11-03 11:14:26.922562 7fb261b4e700 client.5562 trim_dentry unlinking dn in-dir in dir 10000000ce1 2011-11-03 11:14:26.922574 7fb261b4e700 client.5562 unlink dir 0x4188a800 'in-dir' dn 0x41879240 inode 0x41878610 2011-11-03 11:14:26.922586 7fb261b4e700 client.5562 unlink inode 0x41878610 parents now 2011-11-03 11:14:26.922607 7fb261b4e700 client.5562 put_inode on 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41878610) 2011-11-03 11:14:26.922620 7fb261b4e700 inode.put on 0x41878610 10000000ce2.head now 0 2011-11-03 11:14:26.922641 7fb261b4e700 client.5562 remove_cap mds.0 on 10000000ce2.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41878610) 2011-11-03 11:14:26.922653 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.922668 7fb261b4e700 client.5562 put_inode deleting 10000000ce2.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=- 0x41878610) 2011-11-03 11:14:26.922682 7fb261b4e700 client.5562 close_dir dir 0x4188a530 on 0x4188a800 2011-11-03 11:14:26.922705 7fb261b4e700 client.5562 put_inode on 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41879590 0x4188a800) 2011-11-03 11:14:26.922718 7fb261b4e700 inode.put on 0x4188a800 10000000ce1.head now 1 2011-11-03 11:14:26.922730 7fb261b4e700 client.5562 trim_dentry unlinking dn nwatkins in dir 10000000ce0 2011-11-03 11:14:26.922743 7fb261b4e700 client.5562 unlink dir 0x4195cbb0 'nwatkins' dn 0x41879590 inode 0x4188a800 2011-11-03 11:14:26.922760 7fb261b4e700 client.5562 unlink inode 0x4188a800 parents now 2011-11-03 11:14:26.922789 7fb261b4e700 client.5562 put_inode on 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) 0x4188a800) 2011-11-03 11:14:26.922802 7fb261b4e700 inode.put on 0x4188a800 10000000ce1.head now 0 2011-11-03 11:14:26.922823 7fb261b4e700 client.5562 remove_cap mds.0 on 10000000ce1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) 0x4188a800) 2011-11-03 11:14:26.922836 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.922851 7fb261b4e700 client.5562 put_inode deleting 10000000ce1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=- 0x4188a800) 2011-11-03 11:14:26.922865 7fb261b4e700 client.5562 close_dir dir 0x4187a5b0 on 0x4195cbb0 2011-11-03 11:14:26.922888 7fb261b4e700 client.5562 put_inode on 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) parents=0x41b68450 0x4195cbb0) 2011-11-03 11:14:26.922901 7fb261b4e700 inode.put on 0x4195cbb0 10000000ce0.head now 1 2011-11-03 11:14:26.922913 7fb261b4e700 client.5562 trim_dentry unlinking dn user in dir 1 2011-11-03 11:14:26.922926 7fb261b4e700 client.5562 unlink dir 0x41b66000 'user' dn 0x41b68450 inode 0x4195cbb0 2011-11-03 11:14:26.922938 7fb261b4e700 client.5562 unlink inode 0x4195cbb0 parents now 2011-11-03 11:14:26.922958 7fb261b4e700 client.5562 put_inode on 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) 0x4195cbb0) 2011-11-03 11:14:26.922971 7fb261b4e700 inode.put on 0x4195cbb0 10000000ce0.head now 0 2011-11-03 11:14:26.922991 7fb261b4e700 client.5562 remove_cap mds.0 on 10000000ce0.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXs(0=pAsLsXs) 0x4195cbb0) 2011-11-03 11:14:26.923004 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.923019 7fb261b4e700 client.5562 put_inode deleting 10000000ce0.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=- 0x4195cbb0) 2011-11-03 11:14:26.923033 7fb261b4e700 client.5562 close_dir dir 0x41899950 on 0x41b66000 2011-11-03 11:14:26.923055 7fb261b4e700 client.5562 put_inode on 1.head(ref=1 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) 2011-11-03 11:14:26.923067 7fb261b4e700 inode.put on 0x41b66000 1.head now 0 2011-11-03 11:14:26.923088 7fb261b4e700 client.5562 remove_cap mds.0 on 1.head(ref=0 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41b66000) 2011-11-03 11:14:26.923101 7fb261b4e700 client.5562 remove_cap last one, closing snaprealm 0x418997a0 2011-11-03 11:14:26.923116 7fb261b4e700 client.5562 put_inode deleting 1.head(ref=0 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=- 0x41b66000) 2011-11-03 11:14:26.923131 7fb261b4e700 client.5562 flush_caps 2011-11-03 11:14:26.923144 7fb261b4e700 client.5562 wait_sync_caps want 0 (last is 0, 0 total flushing) 2011-11-03 11:14:26.923157 7fb261b4e700 client.5562 sending client_session close to mds.0 seq 3 2011-11-03 11:14:26.923179 7fb261b4e700 -- 192.168.141.130:0/2011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_close seq 3) v1 -- ?+0 0x41c341b0 2011-11-03 11:14:26.923199 7fb261b4e700 -- 192.168.141.130:0/2011511 submit_message client_session(request_close seq 3) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:26.923221 7fb261b4e700 client.5562 waiting for 1 mds sessions to close 2011-11-03 11:14:26.923253 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:26.923283 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer encoding 24 0x41c341b0 client_session(request_close seq 3) v1 2011-11-03 11:14:26.923347 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sending 24 0x41c341b0 2011-11-03 11:14:26.923367 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_message 0x41c341b0 2011-11-03 11:14:26.923401 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:26.923418 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:27.130013 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got ACK 2011-11-03 11:14:27.130055 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:27.177023 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got MSG 2011-11-03 11:14:27.177052 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:14:27.177070 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:14:27.177093 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got front 28 2011-11-03 11:14:27.177116 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).aborted = 0 2011-11-03 11:14:27.177137 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:14:27.177165 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader got message 23 0x418762a0 client_session(close) v1 2011-11-03 11:14:27.177189 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:27.177218 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader reading tag... 2011-11-03 11:14:27.177248 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_entry pipe 0x41c01ea0 dequeued 0x418762a0 2011-11-03 11:14:27.177281 7fb267c69700 -- 192.168.141.130:0/2011511 <== mds.0 192.168.141.123:6802/19591 23 ==== client_session(close) v1 ==== 28+0+0 (3822250925 0 0) 0x418762a0 con 0x41c02110 2011-11-03 11:14:27.177299 7fb267c69700 client.5562 handle_client_session client_session(close) v1 2011-11-03 11:14:27.177319 7fb267c69700 client.5562 kick_requests for mds.0 2011-11-03 11:14:27.177341 7fb267c69700 client.5562 unmounting: trim pass, size was 0+0 2011-11-03 11:14:27.177359 7fb267c69700 client.5562 trim_cache size 0 max 0 2011-11-03 11:14:27.177374 7fb267c69700 client.5562 unmounting: trim pass, cache now empty, waking unmount() 2011-11-03 11:14:27.177398 7fb267c69700 -- 192.168.141.130:0/2011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:14:27.177416 7fb267c69700 -- 192.168.141.130:0/2011511 done calling dispatch on 0x418762a0 2011-11-03 11:14:27.177445 7fb261b4e700 client.5562 unmounted. 2011-11-03 11:14:27.177474 7fb261b4e700 -- 192.168.141.130:0/2011511 mark_down 0x41879c30 -- 0x41876f30 2011-11-03 11:14:27.177497 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).unregister_pipe 2011-11-03 11:14:27.177520 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).stop 2011-11-03 11:14:27.177560 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:27.177606 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).write_ack 23 2011-11-03 11:14:27.177632 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:27.177653 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer sleeping 2011-11-03 11:14:27.177685 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader couldn't read tag, Success 2011-11-03 11:14:27.177713 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).fault 0: Success 2011-11-03 11:14:27.177733 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).fault already closed|closing 2011-11-03 11:14:27.177758 7fb265f61700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).reader done 2011-11-03 11:14:27.177785 7fb261b4e700 client.5562 shutdown 2011-11-03 11:14:27.177820 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer finishing 2011-11-03 11:14:27.177845 7fb266062700 -- 192.168.141.130:0/2011511 queue_reap 0x41876f30 2011-11-03 11:14:27.177874 7fb266062700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).writer done 2011-11-03 11:14:27.177911 7fb26946c700 -- 192.168.141.130:0/2011511 reaper 2011-11-03 11:14:27.177939 7fb26946c700 -- 192.168.141.130:0/2011511 reaper reaping pipe 0x41876f30 192.168.141.132:6800/3579 2011-11-03 11:14:27.177955 7fb26946c700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).discard_queue 2011-11-03 11:14:27.177973 7fb26946c700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1). dequeued pipe 2011-11-03 11:14:27.177996 7fb26946c700 -- 192.168.141.130:0/2011511 >> 192.168.141.132:6800/3579 pipe(0x41876f30 sd=88 pgs=578 cs=1 l=1).unregister_pipe - not registered 2011-11-03 11:14:27.178029 7fb26946c700 -- 192.168.141.130:0/2011511 reaper reaped pipe 0x41876f30 192.168.141.132:6800/3579 2011-11-03 11:14:27.178047 7fb26946c700 -- 192.168.141.130:0/2011511 reaper deleted pipe 0x41876f30 2011-11-03 11:14:27.178062 7fb26946c700 -- 192.168.141.130:0/2011511 reaper done 2011-11-03 11:14:27.178111 7fb261b4e700 -- 192.168.141.130:0/2011511 shutdown 192.168.141.130:0/2011511 2011-11-03 11:14:27.178131 7fb261b4e700 -- 192.168.141.130:0/2011511 shutdown i am not dispatch, setting stop flag and joining thread. 2011-11-03 11:14:27.178150 7fb261b4e700 client.5562 trim_cache size 0 max 0 2011-11-03 11:14:27.178191 7fb261b4e700 -- 192.168.141.130:0/2011511 wait: everything stopped 2011-11-03 11:14:27.178205 7fb261b4e700 -- 192.168.141.130:0/2011511 wait: stopping reaper thread 2011-11-03 11:14:27.178232 7fb26946c700 -- 192.168.141.130:0/2011511 reaper_entry done 2011-11-03 11:14:27.178290 7fb261b4e700 -- 192.168.141.130:0/2011511 wait: stopped reaper thread 2011-11-03 11:14:27.178312 7fb261b4e700 -- 192.168.141.130:0/2011511 wait: closing pipes 2011-11-03 11:14:27.178330 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).unregister_pipe 2011-11-03 11:14:27.178347 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).stop 2011-11-03 11:14:27.178386 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).unregister_pipe 2011-11-03 11:14:27.178409 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).stop 2011-11-03 11:14:27.178448 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader couldn't read tag, Success 2011-11-03 11:14:27.178488 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).fault 0: Success 2011-11-03 11:14:27.178508 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).fault already closed|closing 2011-11-03 11:14:27.178532 7fb266365700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).reader done 2011-11-03 11:14:27.178601 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader couldn't read tag, Success 2011-11-03 11:14:27.178633 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).fault 0: Success 2011-11-03 11:14:27.178654 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).fault already closed|closing 2011-11-03 11:14:27.178678 7fb266163700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).reader done 2011-11-03 11:14:27.178708 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer finishing 2011-11-03 11:14:27.178734 7fb266466700 -- 192.168.141.130:0/2011511 queue_reap 0x41c05220 2011-11-03 11:14:27.178762 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer finishing 2011-11-03 11:14:27.178787 7fb266264700 -- 192.168.141.130:0/2011511 queue_reap 0x41c01ea0 2011-11-03 11:14:27.178811 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper 2011-11-03 11:14:27.178830 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper done 2011-11-03 11:14:27.178845 7fb261b4e700 -- 192.168.141.130:0/2011511 wait: waiting for pipes 0x41c01ea0,0x41c05220 to close 2011-11-03 11:14:27.178884 7fb266466700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).writer done 2011-11-03 11:14:27.178913 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper 2011-11-03 11:14:27.178933 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper reaping pipe 0x41c05220 192.168.141.123:6789/0 2011-11-03 11:14:27.178949 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).discard_queue 2011-11-03 11:14:27.178967 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1). dequeued pipe 2011-11-03 11:14:27.178983 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6789/0 pipe(0x41c05220 sd=83 pgs=380 cs=1 l=1).unregister_pipe - not registered 2011-11-03 11:14:27.179016 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper reaped pipe 0x41c05220 192.168.141.123:6789/0 2011-11-03 11:14:27.179032 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper deleted pipe 0x41c05220 2011-11-03 11:14:27.179045 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper done 2011-11-03 11:14:27.179079 7fb266264700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).writer done 2011-11-03 11:14:27.179108 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper 2011-11-03 11:14:27.179129 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper reaping pipe 0x41c01ea0 192.168.141.123:6802/19591 2011-11-03 11:14:27.179146 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).discard_queue 2011-11-03 11:14:27.179163 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0). dequeued pipe 2011-11-03 11:14:27.179180 7fb261b4e700 -- 192.168.141.130:0/2011511 >> 192.168.141.123:6802/19591 pipe(0x41c01ea0 sd=85 pgs=1179 cs=1 l=0).unregister_pipe - not registered 2011-11-03 11:14:27.179211 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper reaped pipe 0x41c01ea0 192.168.141.123:6802/19591 2011-11-03 11:14:27.179235 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper deleted pipe 0x41c01ea0 2011-11-03 11:14:27.179248 7fb261b4e700 -- 192.168.141.130:0/2011511 reaper done 2011-11-03 11:14:27.179262 7fb261b4e700 -- 192.168.141.130:0/2011511 wait: done. 2011-11-03 11:14:27.179275 7fb261b4e700 -- 192.168.141.130:0/2011511 shutdown complete. 2011-11-03 11:14:27.179368 7fb261b4e700 client.5563 unmounting 2011-11-03 11:14:27.179400 7fb261b4e700 client.5563 put_inode on 1.head(ref=3 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) 2011-11-03 11:14:27.179413 7fb261b4e700 inode.put on 0x41ae6510 1.head now 2 2011-11-03 11:14:27.179426 7fb261b4e700 client.5563 _ll_drop_pins 2011-11-03 11:14:27.179440 7fb261b4e700 client.5563 _ll_put 0x41ae6510 1 1 -> 0 2011-11-03 11:14:27.179461 7fb261b4e700 client.5563 put_inode on 1.head(ref=2 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) 2011-11-03 11:14:27.179473 7fb261b4e700 inode.put on 0x41ae6510 1.head now 1 2011-11-03 11:14:27.179488 7fb261b4e700 client.5563 trim_cache size 12 max 0 2011-11-03 11:14:27.179502 7fb261b4e700 client.5563 trim_dentry unlinking dn job.xml in dir 10000000982 2011-11-03 11:14:27.179515 7fb261b4e700 client.5563 unlink dir 0x41cc53e0 'job.xml' dn 0x41cc6c30 inode 0x41bc45b0 2011-11-03 11:14:27.179527 7fb261b4e700 client.5563 unlink inode 0x41bc45b0 parents now 2011-11-03 11:14:27.179553 7fb261b4e700 client.5563 put_inode on 10000000986.head(ref=1 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] 0x41bc45b0) 2011-11-03 11:14:27.179565 7fb261b4e700 inode.put on 0x41bc45b0 10000000986.head now 0 2011-11-03 11:14:27.179589 7fb261b4e700 client.5563 remove_cap mds.0 on 10000000986.head(ref=0 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] 0x41bc45b0) 2011-11-03 11:14:27.179604 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.179622 7fb261b4e700 client.5563 put_inode deleting 10000000986.head(ref=0 cap_refs={1024=-1,2048=0,8192=0} open={1=0} mode=100644 size=17110 mtime=2011-11-03 11:14:04.597264 caps=- objectset[10000000986 ts 0/0 objects 1 dirty_or_tx 0] 0x41bc45b0) 2011-11-03 11:14:27.179645 7fb261b4e700 client.5563 close_dir dir 0x41cc6dd0 on 0x41cc53e0 2011-11-03 11:14:27.179669 7fb261b4e700 client.5563 put_inode on 10000000982.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc5920 0x41cc53e0) 2011-11-03 11:14:27.179682 7fb261b4e700 inode.put on 0x41cc53e0 10000000982.head now 1 2011-11-03 11:14:27.179694 7fb261b4e700 client.5563 trim_dentry unlinking dn job_201111031113_0001 in dir 100000008af 2011-11-03 11:14:27.179707 7fb261b4e700 client.5563 unlink dir 0x41cc48d0 'job_201111031113_0001' dn 0x41cc5920 inode 0x41cc53e0 2011-11-03 11:14:27.179719 7fb261b4e700 client.5563 unlink inode 0x41cc53e0 parents now 2011-11-03 11:14:27.179740 7fb261b4e700 client.5563 put_inode on 10000000982.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) 0x41cc53e0) 2011-11-03 11:14:27.179752 7fb261b4e700 inode.put on 0x41cc53e0 10000000982.head now 0 2011-11-03 11:14:27.179773 7fb261b4e700 client.5563 remove_cap mds.0 on 10000000982.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=pAsLsXs(0=pAsLsXs) 0x41cc53e0) 2011-11-03 11:14:27.179786 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.179801 7fb261b4e700 client.5563 put_inode deleting 10000000982.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:04.597264 caps=- 0x41cc53e0) 2011-11-03 11:14:27.179820 7fb261b4e700 client.5563 close_dir dir 0x41cc59a0 on 0x41cc48d0 2011-11-03 11:14:27.179852 7fb261b4e700 client.5563 put_inode on 100000008af.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc2670 0x41cc48d0) 2011-11-03 11:14:27.179865 7fb261b4e700 inode.put on 0x41cc48d0 100000008af.head now 1 2011-11-03 11:14:27.179878 7fb261b4e700 client.5563 trim_dentry unlinking dn .staging in dir 100000008ae 2011-11-03 11:14:27.179891 7fb261b4e700 client.5563 unlink dir 0x41cc7970 '.staging' dn 0x41cc2670 inode 0x41cc48d0 2011-11-03 11:14:27.179903 7fb261b4e700 client.5563 unlink inode 0x41cc48d0 parents now 2011-11-03 11:14:27.179923 7fb261b4e700 client.5563 put_inode on 100000008af.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc48d0) 2011-11-03 11:14:27.179936 7fb261b4e700 inode.put on 0x41cc48d0 100000008af.head now 0 2011-11-03 11:14:27.179957 7fb261b4e700 client.5563 remove_cap mds.0 on 100000008af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc48d0) 2011-11-03 11:14:27.179970 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.179985 7fb261b4e700 client.5563 put_inode deleting 100000008af.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-03 11:14:02.107013 caps=- 0x41cc48d0) 2011-11-03 11:14:27.180001 7fb261b4e700 client.5563 close_dir dir 0x41cc4d10 on 0x41cc7970 2011-11-03 11:14:27.180025 7fb261b4e700 client.5563 put_inode on 100000008ae.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc7ee0 0x41cc7970) 2011-11-03 11:14:27.180037 7fb261b4e700 inode.put on 0x41cc7970 100000008ae.head now 1 2011-11-03 11:14:27.180050 7fb261b4e700 client.5563 trim_dentry unlinking dn nwatkins in dir 100000008ad 2011-11-03 11:14:27.180062 7fb261b4e700 client.5563 unlink dir 0x41cc6ea0 'nwatkins' dn 0x41cc7ee0 inode 0x41cc7970 2011-11-03 11:14:27.180075 7fb261b4e700 client.5563 unlink inode 0x41cc7970 parents now 2011-11-03 11:14:27.180096 7fb261b4e700 client.5563 put_inode on 100000008ae.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc7970) 2011-11-03 11:14:27.180108 7fb261b4e700 inode.put on 0x41cc7970 100000008ae.head now 0 2011-11-03 11:14:27.180129 7fb261b4e700 client.5563 remove_cap mds.0 on 100000008ae.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc7970) 2011-11-03 11:14:27.180141 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.180156 7fb261b4e700 client.5563 put_inode deleting 100000008ae.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.718516 caps=- 0x41cc7970) 2011-11-03 11:14:27.180170 7fb261b4e700 client.5563 close_dir dir 0x41cc7f60 on 0x41cc6ea0 2011-11-03 11:14:27.180194 7fb261b4e700 client.5563 put_inode on 100000008ad.head(ref=2 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae61f0 0x41cc6ea0) 2011-11-03 11:14:27.180206 7fb261b4e700 inode.put on 0x41cc6ea0 100000008ad.head now 1 2011-11-03 11:14:27.180219 7fb261b4e700 client.5563 trim_dentry unlinking dn staging in dir 1000000005e 2011-11-03 11:14:27.180231 7fb261b4e700 client.5563 unlink dir 0x41cc6090 'staging' dn 0x41ae61f0 inode 0x41cc6ea0 2011-11-03 11:14:27.180244 7fb261b4e700 client.5563 unlink inode 0x41cc6ea0 parents now 2011-11-03 11:14:27.180264 7fb261b4e700 client.5563 put_inode on 100000008ad.head(ref=1 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc6ea0) 2011-11-03 11:14:27.180277 7fb261b4e700 inode.put on 0x41cc6ea0 100000008ad.head now 0 2011-11-03 11:14:27.180298 7fb261b4e700 client.5563 remove_cap mds.0 on 100000008ad.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc6ea0) 2011-11-03 11:14:27.180317 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.180339 7fb261b4e700 client.5563 put_inode deleting 100000008ad.head(ref=0 cap_refs={} open={} mode=40700 size=0 mtime=2011-11-01 21:07:33.717434 caps=- 0x41cc6ea0) 2011-11-03 11:14:27.180354 7fb261b4e700 client.5563 close_dir dir 0x41cc72e0 on 0x41cc6090 2011-11-03 11:14:27.180377 7fb261b4e700 client.5563 put_inode on 1000000005e.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) parents=0x41cc3eb0 0x41cc6090) 2011-11-03 11:14:27.180390 7fb261b4e700 inode.put on 0x41cc6090 1000000005e.head now 1 2011-11-03 11:14:27.180402 7fb261b4e700 client.5563 trim_dentry unlinking dn mapred in dir 1000000005d 2011-11-03 11:14:27.180415 7fb261b4e700 client.5563 unlink dir 0x41cc3300 'mapred' dn 0x41cc3eb0 inode 0x41cc6090 2011-11-03 11:14:27.180427 7fb261b4e700 client.5563 unlink inode 0x41cc6090 parents now 2011-11-03 11:14:27.180447 7fb261b4e700 client.5563 put_inode on 1000000005e.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) 0x41cc6090) 2011-11-03 11:14:27.180459 7fb261b4e700 inode.put on 0x41cc6090 1000000005e.head now 0 2011-11-03 11:14:27.180480 7fb261b4e700 client.5563 remove_cap mds.0 on 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=pAsLsXs(0=pAsLsXs) 0x41cc6090) 2011-11-03 11:14:27.180493 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.180508 7fb261b4e700 client.5563 put_inode deleting 1000000005e.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:13:37.993864 caps=- 0x41cc6090) 2011-11-03 11:14:27.180521 7fb261b4e700 client.5563 close_dir dir 0x41cc65a0 on 0x41cc3300 2011-11-03 11:14:27.180545 7fb261b4e700 client.5563 put_inode on 1000000005d.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae5e50 0x41cc3300) 2011-11-03 11:14:27.180558 7fb261b4e700 inode.put on 0x41cc3300 1000000005d.head now 1 2011-11-03 11:14:27.180570 7fb261b4e700 client.5563 trim_dentry unlinking dn hadoop-nwatkins in dir 1000000005c 2011-11-03 11:14:27.180583 7fb261b4e700 client.5563 unlink dir 0x41cc28b0 'hadoop-nwatkins' dn 0x41ae5e50 inode 0x41cc3300 2011-11-03 11:14:27.180594 7fb261b4e700 client.5563 unlink inode 0x41cc3300 parents now 2011-11-03 11:14:27.180615 7fb261b4e700 client.5563 put_inode on 1000000005d.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc3300) 2011-11-03 11:14:27.180628 7fb261b4e700 inode.put on 0x41cc3300 1000000005d.head now 0 2011-11-03 11:14:27.180649 7fb261b4e700 client.5563 remove_cap mds.0 on 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc3300) 2011-11-03 11:14:27.180662 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.180677 7fb261b4e700 client.5563 put_inode deleting 1000000005d.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505963 caps=- 0x41cc3300) 2011-11-03 11:14:27.180690 7fb261b4e700 client.5563 close_dir dir 0x41cc37a0 on 0x41cc28b0 2011-11-03 11:14:27.180714 7fb261b4e700 client.5563 put_inode on 1000000005c.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41ae4050 0x41cc28b0) 2011-11-03 11:14:27.180727 7fb261b4e700 inode.put on 0x41cc28b0 1000000005c.head now 1 2011-11-03 11:14:27.180739 7fb261b4e700 client.5563 trim_dentry unlinking dn tmp in dir 1 2011-11-03 11:14:27.180752 7fb261b4e700 client.5563 unlink dir 0x41ae6510 'tmp' dn 0x41ae4050 inode 0x41cc28b0 2011-11-03 11:14:27.180764 7fb261b4e700 client.5563 unlink inode 0x41cc28b0 parents now 2011-11-03 11:14:27.180785 7fb261b4e700 client.5563 put_inode on 1000000005c.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc28b0) 2011-11-03 11:14:27.180810 7fb261b4e700 inode.put on 0x41cc28b0 1000000005c.head now 0 2011-11-03 11:14:27.180831 7fb261b4e700 client.5563 remove_cap mds.0 on 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41cc28b0) 2011-11-03 11:14:27.180844 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.180860 7fb261b4e700 client.5563 put_inode deleting 1000000005c.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:06:06.505221 caps=- 0x41cc28b0) 2011-11-03 11:14:27.180875 7fb261b4e700 client.5563 trim_dentry unlinking dn _partition.lst in dir 10000000ce2 2011-11-03 11:14:27.180888 7fb261b4e700 client.5563 unlink dir 0x41bc5610 '_partition.lst' dn 0x41bc5dd0 inode 0x41bc8660 2011-11-03 11:14:27.180900 7fb261b4e700 client.5563 unlink inode 0x41bc8660 parents now 2011-11-03 11:14:27.180923 7fb261b4e700 client.5563 put_inode on 10000000981.head(ref=1 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x41bc8660) 2011-11-03 11:14:27.180936 7fb261b4e700 inode.put on 0x41bc8660 10000000981.head now 0 2011-11-03 11:14:27.180957 7fb261b4e700 client.5563 remove_cap mds.0 on 10000000981.head(ref=0 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x41bc8660) 2011-11-03 11:14:27.180972 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.180988 7fb261b4e700 client.5563 put_inode deleting 10000000981.head(ref=0 cap_refs={} open={1=0} mode=100777 size=129 mtime=2011-11-03 11:14:01.196477 caps=- objectset[10000000981 ts 0/0 objects 0 dirty_or_tx 0] 0x41bc8660) 2011-11-03 11:14:27.181003 7fb261b4e700 client.5563 close_dir dir 0x41cc5a20 on 0x41bc5610 2011-11-03 11:14:27.181027 7fb261b4e700 client.5563 put_inode on 10000000ce2.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x417a6e70 0x41bc5610) 2011-11-03 11:14:27.181039 7fb261b4e700 inode.put on 0x41bc5610 10000000ce2.head now 1 2011-11-03 11:14:27.181052 7fb261b4e700 client.5563 trim_dentry unlinking dn in-dir in dir 10000000ce1 2011-11-03 11:14:27.181065 7fb261b4e700 client.5563 unlink dir 0x417a6870 'in-dir' dn 0x417a6e70 inode 0x41bc5610 2011-11-03 11:14:27.181077 7fb261b4e700 client.5563 unlink inode 0x41bc5610 parents now 2011-11-03 11:14:27.181098 7fb261b4e700 client.5563 put_inode on 10000000ce2.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41bc5610) 2011-11-03 11:14:27.181110 7fb261b4e700 inode.put on 0x41bc5610 10000000ce2.head now 0 2011-11-03 11:14:27.181131 7fb261b4e700 client.5563 remove_cap mds.0 on 10000000ce2.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41bc5610) 2011-11-03 11:14:27.181144 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.181159 7fb261b4e700 client.5563 put_inode deleting 10000000ce2.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:01.196477 caps=- 0x41bc5610) 2011-11-03 11:14:27.181173 7fb261b4e700 client.5563 close_dir dir 0x41bc5ab0 on 0x417a6870 2011-11-03 11:14:27.181197 7fb261b4e700 client.5563 put_inode on 10000000ce1.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41bc6f80 0x417a6870) 2011-11-03 11:14:27.181210 7fb261b4e700 inode.put on 0x417a6870 10000000ce1.head now 1 2011-11-03 11:14:27.181222 7fb261b4e700 client.5563 trim_dentry unlinking dn nwatkins in dir 10000000ce0 2011-11-03 11:14:27.181234 7fb261b4e700 client.5563 unlink dir 0x417a6430 'nwatkins' dn 0x41bc6f80 inode 0x417a6870 2011-11-03 11:14:27.181252 7fb261b4e700 client.5563 unlink inode 0x417a6870 parents now 2011-11-03 11:14:27.181273 7fb261b4e700 client.5563 put_inode on 10000000ce1.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) 0x417a6870) 2011-11-03 11:14:27.181293 7fb261b4e700 inode.put on 0x417a6870 10000000ce1.head now 0 2011-11-03 11:14:27.181314 7fb261b4e700 client.5563 remove_cap mds.0 on 10000000ce1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=pAsLsXsFs(0=pAsLsXsFs) 0x417a6870) 2011-11-03 11:14:27.181327 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.181342 7fb261b4e700 client.5563 put_inode deleting 10000000ce1.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-03 11:14:05.347177 caps=- 0x417a6870) 2011-11-03 11:14:27.181356 7fb261b4e700 client.5563 close_dir dir 0x417a6dc0 on 0x417a6430 2011-11-03 11:14:27.181380 7fb261b4e700 client.5563 put_inode on 10000000ce0.head(ref=2 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) parents=0x41cc47d0 0x417a6430) 2011-11-03 11:14:27.181392 7fb261b4e700 inode.put on 0x417a6430 10000000ce0.head now 1 2011-11-03 11:14:27.181405 7fb261b4e700 client.5563 trim_dentry unlinking dn user in dir 1 2011-11-03 11:14:27.181417 7fb261b4e700 client.5563 unlink dir 0x41ae6510 'user' dn 0x41cc47d0 inode 0x417a6430 2011-11-03 11:14:27.181429 7fb261b4e700 client.5563 unlink inode 0x417a6430 parents now 2011-11-03 11:14:27.181450 7fb261b4e700 client.5563 put_inode on 10000000ce0.head(ref=1 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) 0x417a6430) 2011-11-03 11:14:27.181462 7fb261b4e700 inode.put on 0x417a6430 10000000ce0.head now 0 2011-11-03 11:14:27.181483 7fb261b4e700 client.5563 remove_cap mds.0 on 10000000ce0.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=pAsLsXsFs(0=pAsLsXsFs) 0x417a6430) 2011-11-03 11:14:27.181496 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.181511 7fb261b4e700 client.5563 put_inode deleting 10000000ce0.head(ref=0 cap_refs={} open={} mode=40777 size=0 mtime=2011-11-01 21:12:56.017215 caps=- 0x417a6430) 2011-11-03 11:14:27.181525 7fb261b4e700 client.5563 close_dir dir 0x41ae3fd0 on 0x41ae6510 2011-11-03 11:14:27.181548 7fb261b4e700 client.5563 put_inode on 1.head(ref=1 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) 2011-11-03 11:14:27.181560 7fb261b4e700 inode.put on 0x41ae6510 1.head now 0 2011-11-03 11:14:27.181581 7fb261b4e700 client.5563 remove_cap mds.0 on 1.head(ref=0 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=pAsLsXsFs(0=pAsLsXsFs) 0x41ae6510) 2011-11-03 11:14:27.181594 7fb261b4e700 client.5563 remove_cap last one, closing snaprealm 0x41ae6430 2011-11-03 11:14:27.181609 7fb261b4e700 client.5563 put_inode deleting 1.head(ref=0 cap_refs={} open={} mode=40755 size=0 mtime=2011-11-01 21:12:55.797267 caps=- 0x41ae6510) 2011-11-03 11:14:27.181623 7fb261b4e700 client.5563 flush_caps 2011-11-03 11:14:27.181635 7fb261b4e700 client.5563 wait_sync_caps want 0 (last is 0, 0 total flushing) 2011-11-03 11:14:27.181648 7fb261b4e700 client.5563 sending client_session close to mds.0 seq 0 2011-11-03 11:14:27.181671 7fb261b4e700 -- 192.168.141.130:0/3011511 --> mds.0 192.168.141.123:6802/19591 -- client_session(request_close) v1 -- ?+0 0x41c003b0 2011-11-03 11:14:27.181689 7fb261b4e700 -- 192.168.141.130:0/3011511 submit_message client_session(request_close) v1 remote, 192.168.141.123:6802/19591, have pipe. 2011-11-03 11:14:27.181707 7fb261b4e700 client.5563 waiting for 1 mds sessions to close 2011-11-03 11:14:27.181743 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:27.181773 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer encoding 21 0x41c003b0 client_session(request_close) v1 2011-11-03 11:14:27.181829 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sending 21 0x41c003b0 2011-11-03 11:14:27.181849 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_message 0x41c003b0 2011-11-03 11:14:27.181881 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:27.181898 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:27.390625 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got ACK 2011-11-03 11:14:27.390665 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:27.444659 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got MSG 2011-11-03 11:14:27.444688 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got envelope type=22 src mds.0 front=28 data=0 off 0 2011-11-03 11:14:27.444705 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader wants 28 from dispatch throttler 0/104857600 2011-11-03 11:14:27.444730 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got front 28 2011-11-03 11:14:27.444752 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).aborted = 0 2011-11-03 11:14:27.444768 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got 28 + 0 + 0 byte message 2011-11-03 11:14:27.444800 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader got message 17 0x41c003b0 client_session(close) v1 2011-11-03 11:14:27.444824 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).queue_received queuing pipe 2011-11-03 11:14:27.444849 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader reading tag... 2011-11-03 11:14:27.444880 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:27.444908 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).write_ack 17 2011-11-03 11:14:27.444938 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_entry pipe 0x7fb27061cc80 dequeued 0x41c003b0 2011-11-03 11:14:27.444969 7fb263e5c700 -- 192.168.141.130:0/3011511 <== mds.0 192.168.141.123:6802/19591 17 ==== client_session(close) v1 ==== 28+0+0 (3822250925 0 0) 0x41c003b0 con 0x7fb27061cef0 2011-11-03 11:14:27.444986 7fb263e5c700 client.5563 handle_client_session client_session(close) v1 2011-11-03 11:14:27.445004 7fb263e5c700 client.5563 kick_requests for mds.0 2011-11-03 11:14:27.445031 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer: state = 2 policy.server=0 2011-11-03 11:14:27.445055 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer sleeping 2011-11-03 11:14:27.445081 7fb263e5c700 client.5563 unmounting: trim pass, size was 0+0 2011-11-03 11:14:27.445098 7fb263e5c700 client.5563 trim_cache size 0 max 0 2011-11-03 11:14:27.445110 7fb263e5c700 client.5563 unmounting: trim pass, cache now empty, waking unmount() 2011-11-03 11:14:27.445136 7fb263e5c700 -- 192.168.141.130:0/3011511 dispatch_throttle_release 28 to dispatch throttler 28/104857600 2011-11-03 11:14:27.445162 7fb263e5c700 -- 192.168.141.130:0/3011511 done calling dispatch on 0x41c003b0 2011-11-03 11:14:27.445185 7fb261b4e700 client.5563 unmounted. 2011-11-03 11:14:27.445207 7fb261b4e700 -- 192.168.141.130:0/3011511 mark_down 0x7fb270736880 -- 0x7fb270736000 2011-11-03 11:14:27.445225 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).unregister_pipe 2011-11-03 11:14:27.445243 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).stop 2011-11-03 11:14:27.445289 7fb261b4e700 client.5563 shutdown 2011-11-03 11:14:27.445323 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer finishing 2011-11-03 11:14:27.445352 7fb262255700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).writer done 2011-11-03 11:14:27.445387 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader couldn't read tag, Success 2011-11-03 11:14:27.445414 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).fault 0: Success 2011-11-03 11:14:27.445432 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).fault already closed|closing 2011-11-03 11:14:27.445453 7fb262154700 -- 192.168.141.130:0/3011511 queue_reap 0x7fb270736000 2011-11-03 11:14:27.445478 7fb262154700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).reader done 2011-11-03 11:14:27.445541 7fb26565f700 -- 192.168.141.130:0/3011511 reaper 2011-11-03 11:14:27.445566 7fb26565f700 -- 192.168.141.130:0/3011511 reaper reaping pipe 0x7fb270736000 192.168.141.132:6800/3579 2011-11-03 11:14:27.445585 7fb26565f700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).discard_queue 2011-11-03 11:14:27.445607 7fb26565f700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1). dequeued pipe 2011-11-03 11:14:27.445628 7fb26565f700 -- 192.168.141.130:0/3011511 >> 192.168.141.132:6800/3579 pipe(0x7fb270736000 sd=92 pgs=579 cs=1 l=1).unregister_pipe - not registered 2011-11-03 11:14:27.445657 7fb261b4e700 -- 192.168.141.130:0/3011511 shutdown 192.168.141.130:0/3011511 2011-11-03 11:14:27.445680 7fb261b4e700 -- 192.168.141.130:0/3011511 shutdown i am not dispatch, setting stop flag and joining thread. 2011-11-03 11:14:27.445706 7fb26565f700 -- 192.168.141.130:0/3011511 reaper reaped pipe 0x7fb270736000 192.168.141.132:6800/3579 2011-11-03 11:14:27.445730 7fb26565f700 -- 192.168.141.130:0/3011511 reaper deleted pipe 0x7fb270736000 2011-11-03 11:14:27.445747 7fb26565f700 -- 192.168.141.130:0/3011511 reaper done 2011-11-03 11:14:27.445769 7fb261b4e700 client.5563 trim_cache size 0 max 0 2011-11-03 11:14:27.445828 7fb261b4e700 -- 192.168.141.130:0/3011511 wait: everything stopped 2011-11-03 11:14:27.445843 7fb261b4e700 -- 192.168.141.130:0/3011511 wait: stopping reaper thread 2011-11-03 11:14:27.445869 7fb26565f700 -- 192.168.141.130:0/3011511 reaper_entry done 2011-11-03 11:14:27.445922 7fb261b4e700 -- 192.168.141.130:0/3011511 wait: stopped reaper thread 2011-11-03 11:14:27.445941 7fb261b4e700 -- 192.168.141.130:0/3011511 wait: closing pipes 2011-11-03 11:14:27.445958 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).unregister_pipe 2011-11-03 11:14:27.445975 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).stop 2011-11-03 11:14:27.446014 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).unregister_pipe 2011-11-03 11:14:27.446043 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).stop 2011-11-03 11:14:27.446083 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer finishing 2011-11-03 11:14:27.446114 7fb262659700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).writer done 2011-11-03 11:14:27.446143 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader couldn't read tag, Success 2011-11-03 11:14:27.446173 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).fault 0: Success 2011-11-03 11:14:27.446192 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).fault already closed|closing 2011-11-03 11:14:27.446238 7fb262356700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).reader done 2011-11-03 11:14:27.446270 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader couldn't read tag, Success 2011-11-03 11:14:27.446298 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).fault 0: Success 2011-11-03 11:14:27.446319 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).fault already closed|closing 2011-11-03 11:14:27.446340 7fb262558700 -- 192.168.141.130:0/3011511 queue_reap 0x41cc1130 2011-11-03 11:14:27.446371 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer finishing 2011-11-03 11:14:27.446400 7fb262457700 -- 192.168.141.130:0/3011511 queue_reap 0x7fb27061cc80 2011-11-03 11:14:27.446424 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper 2011-11-03 11:14:27.446442 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper done 2011-11-03 11:14:27.446457 7fb261b4e700 -- 192.168.141.130:0/3011511 wait: waiting for pipes 0x41cc1130,0x7fb27061cc80 to close 2011-11-03 11:14:27.446495 7fb262558700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).reader done 2011-11-03 11:14:27.446523 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper 2011-11-03 11:14:27.446544 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper reaping pipe 0x41cc1130 192.168.141.123:6789/0 2011-11-03 11:14:27.446560 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).discard_queue 2011-11-03 11:14:27.446578 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1). dequeued pipe 2011-11-03 11:14:27.446595 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6789/0 pipe(0x41cc1130 sd=87 pgs=381 cs=1 l=1).unregister_pipe - not registered 2011-11-03 11:14:27.446634 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper reaped pipe 0x41cc1130 192.168.141.123:6789/0 2011-11-03 11:14:27.446651 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper deleted pipe 0x41cc1130 2011-11-03 11:14:27.446664 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper done 2011-11-03 11:14:27.446697 7fb262457700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).writer done 2011-11-03 11:14:27.446725 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper 2011-11-03 11:14:27.446745 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper reaping pipe 0x7fb27061cc80 192.168.141.123:6802/19591 2011-11-03 11:14:27.446762 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).discard_queue 2011-11-03 11:14:27.446780 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0). dequeued pipe 2011-11-03 11:14:27.446802 7fb261b4e700 -- 192.168.141.130:0/3011511 >> 192.168.141.123:6802/19591 pipe(0x7fb27061cc80 sd=89 pgs=1180 cs=1 l=0).unregister_pipe - not registered 2011-11-03 11:14:27.446835 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper reaped pipe 0x7fb27061cc80 192.168.141.123:6802/19591 2011-11-03 11:14:27.446853 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper deleted pipe 0x7fb27061cc80 2011-11-03 11:14:27.446867 7fb261b4e700 -- 192.168.141.130:0/3011511 reaper done 2011-11-03 11:14:27.446880 7fb261b4e700 -- 192.168.141.130:0/3011511 wait: done. 2011-11-03 11:14:27.446893 7fb261b4e700 -- 192.168.141.130:0/3011511 shutdown complete.